[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:44:11 dist-test-slave-5w2x krb5kdc[16708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:44:11 dist-test-slave-5w2x krb5kdc[16708](info): set up 2 sockets
Jun 19 06:44:11 dist-test-slave-5w2x krb5kdc[16708](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 06:44:12.529697 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.004s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:44:12 dist-test-slave-5w2x krb5kdc[16708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:44:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:44:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:44:12.695840 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:42155
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:42155
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:13.009531 16721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:13.009876 16721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:13.010044 16721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:13.017886 16721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:44:13.018039 16721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:13.018122 16721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:13.018206 16721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:44:13.018294 16721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:44:13.028035 16721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:42155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:42155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:13.029403 16721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:13.030730 16721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:13.042769 16729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:13.042878 16726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:13.042790 16727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:13.043243 16721 server_base.cc:1034] running on GCE node
I20260619 06:44:13.044292 16721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:13.045750 16721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:13.046947 16721 hybrid_clock.cc:648] HybridClock initialized: now 1781851453046843 us; error 99 us; skew 500 ppm
I20260619 06:44:13.047299 16721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:13 dist-test-slave-5w2x krb5kdc[16708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851453, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:13.050292 16721 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:44:13.051635 16721 webserver.cc:460] Webserver started at http://127.16.79.126:46153/ using document root <none> and password file <none>
I20260619 06:44:13.052338 16721 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:13.052438 16721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:13.052691 16721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:13.055464 16721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "dac5cb26bc23460294688539a67d3761"
format_stamp: "Formatted at 2026-06-19 06:44:13 on dist-test-slave-5w2x"
server_key: "20172c549e2ef0af7876d64117cfae6c"
server_key_iv: "77c3bb13b746505bec805cb20a26ca7e"
server_key_version: "encryptionkey@0"
I20260619 06:44:13.056218 16721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "dac5cb26bc23460294688539a67d3761"
format_stamp: "Formatted at 2026-06-19 06:44:13 on dist-test-slave-5w2x"
server_key: "20172c549e2ef0af7876d64117cfae6c"
server_key_iv: "77c3bb13b746505bec805cb20a26ca7e"
server_key_version: "encryptionkey@0"
I20260619 06:44:13.061224 16721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 06:44:13.064607 16736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:13.066169 16721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:44:13.066336 16721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "dac5cb26bc23460294688539a67d3761"
format_stamp: "Formatted at 2026-06-19 06:44:13 on dist-test-slave-5w2x"
server_key: "20172c549e2ef0af7876d64117cfae6c"
server_key_iv: "77c3bb13b746505bec805cb20a26ca7e"
server_key_version: "encryptionkey@0"
I20260619 06:44:13.066619 16721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:13.084280 16721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:13.088110 16721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:13.088428 16721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:13.106287 16721 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:42155
I20260619 06:44:13.106294 16787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:42155 every 8 connection(s)
I20260619 06:44:13.107692 16721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:44:13.111344 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 16721
I20260619 06:44:13.111680 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:44:13.111807 16788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:13.114106 16701 external_mini_cluster.cc:1428] Setting key 0a3d067eb404da85525cfc6b3de58446
I20260619 06:44:13.123769 16788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761: Bootstrap starting.
I20260619 06:44:13.126780 16788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:13.127810 16788 log.cc:824] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:13.131191 16788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761: No bootstrap required, opened a new log
I20260619 06:44:13.139896 16788 raft_consensus.cc:348] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dac5cb26bc23460294688539a67d3761" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42155 } }
I20260619 06:44:13.140389 16788 raft_consensus.cc:374] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:13.140496 16788 raft_consensus.cc:729] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dac5cb26bc23460294688539a67d3761, State: Initialized, Role: FOLLOWER
I20260619 06:44:13.141098 16788 consensus_queue.cc:260] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dac5cb26bc23460294688539a67d3761" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42155 } }
I20260619 06:44:13.141333 16788 raft_consensus.cc:388] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:44:13.141458 16788 raft_consensus.cc:482] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:44:13.141624 16788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [term 0 FOLLOWER]: Advancing to term 1
Jun 19 06:44:13 dist-test-slave-5w2x krb5kdc[16708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:13.143790 16788 raft_consensus.cc:504] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dac5cb26bc23460294688539a67d3761" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42155 } }
I20260619 06:44:13.144357 16788 leader_election.cc:302] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dac5cb26bc23460294688539a67d3761; no voters:
I20260619 06:44:13.145040 16788 leader_election.cc:288] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:44:13.145186 16793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:13.146932 16793 raft_consensus.cc:686] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [term 1 LEADER]: Becoming Leader. State: Replica: dac5cb26bc23460294688539a67d3761, State: Running, Role: LEADER
I20260619 06:44:13.147346 16788 sys_catalog.cc:564] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:44:13.147557 16793 consensus_queue.cc:237] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dac5cb26bc23460294688539a67d3761" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42155 } }
I20260619 06:44:13.150700 16794 sys_catalog.cc:455] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dac5cb26bc23460294688539a67d3761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dac5cb26bc23460294688539a67d3761" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42155 } } }
I20260619 06:44:13.151147 16794 sys_catalog.cc:458] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:13.152974 16795 sys_catalog.cc:455] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dac5cb26bc23460294688539a67d3761. Latest consensus state: current_term: 1 leader_uuid: "dac5cb26bc23460294688539a67d3761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dac5cb26bc23460294688539a67d3761" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42155 } } }
I20260619 06:44:13.153733 16795 sys_catalog.cc:458] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761 [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:13.159018 16790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:13.118627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56906 (local address 127.16.79.126:42155)
0619 06:44:13.119516 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:13.119531 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:13.120855 (+ 1324us) server_negotiation.cc:407] Connection header received
0619 06:44:13.130077 (+ 9222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:13.130102 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:13.130578 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:13.131003 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:13.133827 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:13.135849 (+ 2022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:13.137475 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:13.138557 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:13.138756 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:13.144669 (+ 5913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:13.144681 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:13.144699 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:13.144753 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:13.149355 (+ 4602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:13.151144 (+ 1789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:13.151151 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:13.151157 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:13.151395 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:13.151998 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:13.152004 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:13.152007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:13.152844 (+ 837us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:13.156341 (+ 3497us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:13.156673 (+ 332us) server_negotiation.cc:299] Negotiation successful
0619 06:44:13.157178 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":394,"threads_started":1}
I20260619 06:44:13.159850 16799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:44:13.165253 16799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:44:13.179003 16799 catalog_manager.cc:1347] Generated new cluster ID: e5bfc5d49bc14ea7ae55e2bc92705c8c
I20260619 06:44:13.179139 16799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:44:13.192108 16799 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:44:13.193356 16799 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:44:13.203372 16799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dac5cb26bc23460294688539a67d3761: Generated new TSK 0
I20260619 06:44:13.204337 16799 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:44:13.250254 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42155
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:13.578862 16814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:13.579201 16814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:13.579370 16814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:13.586710 16814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:13.586889 16814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:13.587038 16814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:44:13.596827 16814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42155
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:13.598155 16814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:13.599466 16814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:13.612404 16822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:13.612495 16820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:13.612542 16819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:13.617609 16814 server_base.cc:1034] running on GCE node
I20260619 06:44:13.618244 16814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:13.619340 16814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:13.620584 16814 hybrid_clock.cc:648] HybridClock initialized: now 1781851453620532 us; error 49 us; skew 500 ppm
I20260619 06:44:13.620903 16814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:13 dist-test-slave-5w2x krb5kdc[16708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851453, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:13.624064 16814 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:44:13.625556 16814 webserver.cc:460] Webserver started at http://127.16.79.65:42713/ using document root <none> and password file <none>
I20260619 06:44:13.626289 16814 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:13.626431 16814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:13.626750 16814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:13.629737 16814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "74231b89345c48a7be3d59ffb1eea677"
format_stamp: "Formatted at 2026-06-19 06:44:13 on dist-test-slave-5w2x"
server_key: "fb057e33ef3394180a236691e984a229"
server_key_iv: "0b397f3fb3f8bdc7a317a5830873333f"
server_key_version: "encryptionkey@0"
I20260619 06:44:13.630478 16814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "74231b89345c48a7be3d59ffb1eea677"
format_stamp: "Formatted at 2026-06-19 06:44:13 on dist-test-slave-5w2x"
server_key: "fb057e33ef3394180a236691e984a229"
server_key_iv: "0b397f3fb3f8bdc7a317a5830873333f"
server_key_version: "encryptionkey@0"
I20260619 06:44:13.635506 16814 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:44:13.639066 16829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:13.640444 16814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:44:13.640637 16814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "74231b89345c48a7be3d59ffb1eea677"
format_stamp: "Formatted at 2026-06-19 06:44:13 on dist-test-slave-5w2x"
server_key: "fb057e33ef3394180a236691e984a229"
server_key_iv: "0b397f3fb3f8bdc7a317a5830873333f"
server_key_version: "encryptionkey@0"
I20260619 06:44:13.640843 16814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:13.666517 16814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:13.671041 16814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:13.671324 16814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:13.672413 16814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:13.673724 16814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:13.673831 16814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:13.673970 16814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:13.674036 16814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:13.707664 16814 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:44533
I20260619 06:44:13.707712 16941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:44533 every 8 connection(s)
I20260619 06:44:13.708986 16814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:44:13.719419 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 16814
I20260619 06:44:13.719656 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:44:13.720312 16701 external_mini_cluster.cc:1428] Setting key d12f5419c519be3220094cbbc3ae8803
Jun 19 06:44:13 dist-test-slave-5w2x krb5kdc[16708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851453, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:13.728638 16944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:13.711448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:39423 (local address 127.16.79.126:42155)
0619 06:44:13.711891 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:13.711895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:13.712910 (+ 1015us) server_negotiation.cc:407] Connection header received
0619 06:44:13.714718 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:13.714723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:13.714811 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:13.714929 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:13.716416 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:13.717133 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:13.718699 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:13.719246 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:13.719399 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:13.723148 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:13.723158 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:13.723161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:13.723190 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:13.725687 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:13.726387 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:13.726399 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:13.726401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:13.726516 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:13.726950 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:13.726955 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:13.726958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:13.727220 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:13.727373 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:13.728047 (+ 674us) server_negotiation.cc:299] Negotiation successful
0619 06:44:13.728292 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":243,"threads_started":1}
I20260619 06:44:13.729561 16945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:13.712039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:42155 (local address 127.16.79.65:39423)
0619 06:44:13.712757 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:13.712801 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:13.714448 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:13.715164 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:13.715172 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:13.715652 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:13.716254 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:13.716273 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:13.717291 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:13.717304 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:13.718534 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:13.718550 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:13.719468 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:13.719473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:13.719791 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:13.720514 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:13.720535 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:13.722933 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:13.725825 (+ 2892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:13.725834 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:13.725851 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:13.726261 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:13.726630 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:13.726635 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:13.726639 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:13.726831 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:13.727349 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:13.727356 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:13.727684 (+ 328us) client_negotiation.cc:769] Sending connection context
0619 06:44:13.727943 (+ 259us) client_negotiation.cc:240] Negotiation successful
0619 06:44:13.728484 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":257,"threads_started":1}
I20260619 06:44:13.731081 16942 heartbeater.cc:347] Connected to a master server at 127.16.79.126:42155
I20260619 06:44:13.731442 16942 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:13.732376 16942 heartbeater.cc:511] Master 127.16.79.126:42155 requested a full tablet report, sending...
I20260619 06:44:13.734614 16753 ts_manager.cc:194] Registered new tserver with Master: 74231b89345c48a7be3d59ffb1eea677 (127.16.79.65:44533)
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
I20260619 06:44:13.736119 16753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:39423
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:44:13.757295 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42155
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:14.071574 16948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:14.071995 16948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:14.072197 16948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:14.079977 16948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:14.080174 16948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:14.080380 16948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:44:14.090229 16948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42155
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:14.091549 16948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:14.092943 16948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:14.105769 16953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:14.105738 16956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:14.105696 16954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:14.109396 16948 server_base.cc:1034] running on GCE node
I20260619 06:44:14.110076 16948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:14.111251 16948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:14.112509 16948 hybrid_clock.cc:648] HybridClock initialized: now 1781851454112459 us; error 57 us; skew 500 ppm
I20260619 06:44:14.112871 16948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:14 dist-test-slave-5w2x krb5kdc[16708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851454, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:14.116482 16948 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:44:14.117947 16948 webserver.cc:460] Webserver started at http://127.16.79.66:40663/ using document root <none> and password file <none>
I20260619 06:44:14.118633 16948 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:14.118782 16948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:14.119102 16948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:14.122008 16948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "fc1ea7656b2a4b07af187c812d8e2db4"
format_stamp: "Formatted at 2026-06-19 06:44:14 on dist-test-slave-5w2x"
server_key: "b39fb5f5c31c1b7fc32e6d7efbfdddf5"
server_key_iv: "7da7db4027085d0d48acc81bec9b6ae3"
server_key_version: "encryptionkey@0"
I20260619 06:44:14.122763 16948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "fc1ea7656b2a4b07af187c812d8e2db4"
format_stamp: "Formatted at 2026-06-19 06:44:14 on dist-test-slave-5w2x"
server_key: "b39fb5f5c31c1b7fc32e6d7efbfdddf5"
server_key_iv: "7da7db4027085d0d48acc81bec9b6ae3"
server_key_version: "encryptionkey@0"
I20260619 06:44:14.127830 16948 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:44:14.131276 16963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:14.132625 16948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:44:14.132797 16948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "fc1ea7656b2a4b07af187c812d8e2db4"
format_stamp: "Formatted at 2026-06-19 06:44:14 on dist-test-slave-5w2x"
server_key: "b39fb5f5c31c1b7fc32e6d7efbfdddf5"
server_key_iv: "7da7db4027085d0d48acc81bec9b6ae3"
server_key_version: "encryptionkey@0"
I20260619 06:44:14.132997 16948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:14.149474 16948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:14.153613 16948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:14.153934 16948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:14.155128 16948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:14.156508 16948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:14.156581 16948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:14.156661 16948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:14.156704 16948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:14.190459 16948 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:33419
I20260619 06:44:14.190510 17075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:33419 every 8 connection(s)
I20260619 06:44:14.191773 16948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:44:14.192291 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 16948
I20260619 06:44:14.192518 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:44:14.193101 16701 external_mini_cluster.cc:1428] Setting key 99b59fdfe9363155e9044754d1d7f7df
Jun 19 06:44:14 dist-test-slave-5w2x krb5kdc[16708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851454, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
I20260619 06:44:14.212054 16944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:14.194619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:38653 (local address 127.16.79.126:42155)
0619 06:44:14.194820 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:14.194826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:14.196181 (+ 1355us) server_negotiation.cc:407] Connection header received
0619 06:44:14.197960 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:14.197965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:14.198045 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:14.198170 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:14.199642 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:14.200441 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:14.201933 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:14.202338 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:14.202465 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:14.206005 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:14.206010 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:14.206013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:14.206045 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:14.208783 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:14.209489 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:14.209496 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:14.209499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:14.209627 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:14.210039 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:14.210047 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:14.210050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:14.210397 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:14.210559 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:14.211321 (+ 762us) server_negotiation.cc:299] Negotiation successful
0619 06:44:14.211585 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:44:14.212780 17078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:14.195258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:42155 (local address 127.16.79.66:38653)
0619 06:44:14.196016 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:14.196053 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:14.197691 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:14.198402 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:14.198411 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:14.198912 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:14.199461 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:14.199479 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:14.200580 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:14.200595 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:14.201776 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:14.201787 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:14.202560 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:14.202567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:14.202799 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:14.203506 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:14.203527 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:14.205779 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:14.208925 (+ 3146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:14.208934 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:14.208952 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:14.209365 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:14.209735 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:14.209741 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:14.209745 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:14.209928 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:14.210569 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:14.210578 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:14.210929 (+ 351us) client_negotiation.cc:769] Sending connection context
0619 06:44:14.211181 (+ 252us) client_negotiation.cc:240] Negotiation successful
0619 06:44:14.211603 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":320,"threads_started":1}
I20260619 06:44:14.214346 17076 heartbeater.cc:347] Connected to a master server at 127.16.79.126:42155
I20260619 06:44:14.214668 17076 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:14.215569 17076 heartbeater.cc:511] Master 127.16.79.126:42155 requested a full tablet report, sending...
I20260619 06:44:14.217131 16753 ts_manager.cc:194] Registered new tserver with Master: fc1ea7656b2a4b07af187c812d8e2db4 (127.16.79.66:33419)
I20260619 06:44:14.217921 16753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:38653
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:44:14.231534 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42155
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:14.553419 17081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:14.553716 17081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:14.553876 17081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:14.561226 17081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:14.561383 17081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:14.561565 17081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:44:14.571628 17081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42155
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:14.573027 17081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:14.574359 17081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:14.587649 17089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:14.587659 17087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:14.587654 17086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:14.589196 17081 server_base.cc:1034] running on GCE node
I20260619 06:44:14.589857 17081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:14.590829 17081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:14.592070 17081 hybrid_clock.cc:648] HybridClock initialized: now 1781851454592037 us; error 48 us; skew 500 ppm
I20260619 06:44:14.592422 17081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:14 dist-test-slave-5w2x krb5kdc[16708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851454, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:14.605338 17081 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:44:14.606936 17081 webserver.cc:460] Webserver started at http://127.16.79.67:35323/ using document root <none> and password file <none>
I20260619 06:44:14.607604 17081 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:14.607705 17081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:14.607955 17081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:14.610826 17081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "620e3bc89e204509a7aa7e9e84c51a82"
format_stamp: "Formatted at 2026-06-19 06:44:14 on dist-test-slave-5w2x"
server_key: "6225ecfb4beae494b19bc06ff3bfa078"
server_key_iv: "c0cc94ca29237656ab7e516335ab9115"
server_key_version: "encryptionkey@0"
I20260619 06:44:14.611562 17081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "620e3bc89e204509a7aa7e9e84c51a82"
format_stamp: "Formatted at 2026-06-19 06:44:14 on dist-test-slave-5w2x"
server_key: "6225ecfb4beae494b19bc06ff3bfa078"
server_key_iv: "c0cc94ca29237656ab7e516335ab9115"
server_key_version: "encryptionkey@0"
I20260619 06:44:14.617143 17081 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260619 06:44:14.620851 17096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:14.622365 17081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:44:14.622598 17081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "620e3bc89e204509a7aa7e9e84c51a82"
format_stamp: "Formatted at 2026-06-19 06:44:14 on dist-test-slave-5w2x"
server_key: "6225ecfb4beae494b19bc06ff3bfa078"
server_key_iv: "c0cc94ca29237656ab7e516335ab9115"
server_key_version: "encryptionkey@0"
I20260619 06:44:14.622862 17081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:14.634418 17081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:14.639093 17081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:14.639417 17081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:14.640534 17081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:14.641921 17081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:14.642032 17081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:14.642190 17081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:14.642270 17081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:14.678212 17081 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:42941
I20260619 06:44:14.678220 17208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:42941 every 8 connection(s)
I20260619 06:44:14.679591 17081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:44:14.688546 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 17081
I20260619 06:44:14.688763 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:44:14.689222 16701 external_mini_cluster.cc:1428] Setting key 480fc6d161c0cebe9bb1ea45d9958a52
Jun 19 06:44:14 dist-test-slave-5w2x krb5kdc[16708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851454, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:14.699730 16944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:14.682257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:51903 (local address 127.16.79.126:42155)
0619 06:44:14.682498 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:14.682505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:14.683696 (+ 1191us) server_negotiation.cc:407] Connection header received
0619 06:44:14.685573 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:14.685578 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:14.685666 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:14.685765 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:14.687220 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:14.688049 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:14.689803 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:14.690342 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:14.690453 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:14.694272 (+ 3819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:14.694277 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:14.694281 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:14.694313 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:14.697008 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:14.697678 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:14.697683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:14.697685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:14.697816 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:14.698245 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:14.698248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:14.698250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:14.698554 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:14.698670 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:14.699297 (+ 627us) server_negotiation.cc:299] Negotiation successful
0619 06:44:14.699452 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 06:44:14.700765 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:14.682816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:42155 (local address 127.16.79.67:51903)
0619 06:44:14.683532 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:14.683569 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:14.685301 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:14.686007 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:14.686015 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:14.686510 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:14.687063 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:14.687080 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:14.688250 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:14.688254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:14.689637 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:14.689653 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:14.690557 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:14.690564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:14.690817 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:14.691528 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:14.691549 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:14.694080 (+ 2531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:14.697131 (+ 3051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:14.697138 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:14.697149 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:14.697551 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:14.697913 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:14.697916 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:14.697918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:14.698140 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:14.698669 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:14.698675 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:14.698955 (+ 280us) client_negotiation.cc:769] Sending connection context
0619 06:44:14.699187 (+ 232us) client_negotiation.cc:240] Negotiation successful
0619 06:44:14.699541 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":294,"threads_started":1}
I20260619 06:44:14.702186 17209 heartbeater.cc:347] Connected to a master server at 127.16.79.126:42155
I20260619 06:44:14.702486 17209 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:14.703313 17209 heartbeater.cc:511] Master 127.16.79.126:42155 requested a full tablet report, sending...
I20260619 06:44:14.704991 16753 ts_manager.cc:194] Registered new tserver with Master: 620e3bc89e204509a7aa7e9e84c51a82 (127.16.79.67:42941)
I20260619 06:44:14.705864 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:44:14.706025 16753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:51903
Jun 19 06:44:14 dist-test-slave-5w2x krb5kdc[16708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851454, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260619 06:44:14.739127 16942 heartbeater.cc:503] Master 127.16.79.126:42155 was elected leader, sending a full tablet report...
Jun 19 06:44:14 dist-test-slave-5w2x krb5kdc[16708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851454, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:44:14.750334 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:14.734763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37764 (local address 127.16.79.65:44533)
0619 06:44:14.735424 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:14.735432 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:14.735467 (+ 35us) server_negotiation.cc:407] Connection header received
0619 06:44:14.735552 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:14.735559 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:14.735773 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:14.735942 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:14.737426 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:14.738440 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:14.739781 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:14.740408 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:14.740550 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:14.744553 (+ 4003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:14.744564 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:14.744577 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:14.744612 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:14.747394 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:14.748120 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:14.748127 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:14.748167 (+ 40us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:14.748307 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:14.748713 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:14.748719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:14.748723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:14.749133 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:14.749424 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:14.749726 (+ 302us) server_negotiation.cc:299] Negotiation successful
0619 06:44:14.749978 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":342,"threads_started":1}
W20260619 06:44:14.751580 16857 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37764
Jun 19 06:44:14 dist-test-slave-5w2x krb5kdc[16708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 06:44:14 dist-test-slave-5w2x krb5kdc[16708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:44:14.793808 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:14.780547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37776 (local address 127.16.79.65:44533)
0619 06:44:14.780823 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:14.780830 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:14.781038 (+ 208us) server_negotiation.cc:407] Connection header received
0619 06:44:14.781271 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:14.781277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:14.781411 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:14.781537 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:14.782806 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:14.783485 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:14.784733 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:14.785166 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:14.785271 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:14.788996 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:14.789001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:14.789005 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:14.789038 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:14.791495 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:14.792128 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:14.792132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:14.792189 (+ 57us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:14.792326 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:14.792657 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:14.792661 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:14.792663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:14.792900 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:14.793066 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:14.793317 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 06:44:14.793472 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260619 06:44:14.795274 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 16814
I20260619 06:44:14.807910 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:14.796250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37792 (local address 127.16.79.65:44533)
0619 06:44:14.796434 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:14.796442 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:14.796758 (+ 316us) server_negotiation.cc:407] Connection header received
0619 06:44:14.796932 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:14.796941 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:14.796995 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:14.797094 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:14.798210 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:14.798894 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:14.800186 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:14.800611 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:14.800740 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:14.802959 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:14.802964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:14.802967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:14.802995 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:14.805435 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:14.806060 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:14.806072 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:14.806075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:14.806209 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:14.806569 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:14.806576 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:14.806581 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:14.806831 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:14.807072 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:14.807361 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 06:44:14.807560 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 06:44:14.808699 16937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:14.938182 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 16814
I20260619 06:44:14.963158 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 16948
Jun 19 06:44:14 dist-test-slave-5w2x krb5kdc[16708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:44:14.980446 17223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:14.964251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59340 (local address 127.16.79.66:33419)
0619 06:44:14.964817 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:14.964824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:14.964848 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:44:14.964905 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:14.964911 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:14.965116 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:14.965287 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:14.966470 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:14.967454 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:14.968873 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:14.969403 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:14.969552 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:14.974248 (+ 4696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:14.974259 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:14.974272 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:14.974311 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:14.977323 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:14.978245 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:14.978252 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:14.978255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:14.978380 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:14.978854 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:14.978857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:14.978859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:14.979206 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:14.979421 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:14.979819 (+ 398us) server_negotiation.cc:299] Negotiation successful
0619 06:44:14.980015 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":316,"threads_started":1}
I20260619 06:44:14.981822 17071 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:15.063490 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 16948
I20260619 06:44:15.082636 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 17081
Jun 19 06:44:15 dist-test-slave-5w2x krb5kdc[16708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:44:15.099025 17225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:15.083671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41410 (local address 127.16.79.67:42941)
0619 06:44:15.084388 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:15.084395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:15.084422 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:44:15.084494 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:15.084500 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:15.084745 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:15.084913 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:15.086139 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:15.087158 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:15.088553 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:15.089117 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:15.089270 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:15.093518 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:15.093530 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:15.093580 (+ 50us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:15.093615 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:15.096599 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:15.097222 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:15.097229 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:15.097231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:15.097350 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:15.097654 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:15.097657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:15.097659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:15.097978 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:15.098213 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:15.098435 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 06:44:15.098634 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":370,"threads_started":1}
I20260619 06:44:15.100400 17204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:15.186095 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 17081
I20260619 06:44:15.201635 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 16721
I20260619 06:44:15.202744 16783 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:15.306910 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 16721
2026-06-19T06:44:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3875 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:44:15 dist-test-slave-5w2x krb5kdc[17230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:44:15 dist-test-slave-5w2x krb5kdc[17230](info): set up 2 sockets
Jun 19 06:44:15 dist-test-slave-5w2x krb5kdc[17230](info): commencing operation
krb5kdc: starting...
W20260619 06:44:16.491940 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.004s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:44:16 dist-test-slave-5w2x krb5kdc[17230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:44:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:44:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:44:16.663859 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:44761
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:44761
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:16.974474 17243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:16.974777 17243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:16.974869 17243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:16.982199 17243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:44:16.982311 17243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:16.982357 17243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:16.982407 17243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:44:16.982451 17243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:44:16.992168 17243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:44761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:44761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:16.993392 17243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:16.994616 17243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:17.006055 17248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:17.005993 17249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:17.006018 17251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:17.007133 17243 server_base.cc:1034] running on GCE node
I20260619 06:44:17.007853 17243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:17.009263 17243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:17.010469 17243 hybrid_clock.cc:648] HybridClock initialized: now 1781851457010418 us; error 43 us; skew 500 ppm
I20260619 06:44:17.010847 17243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:17 dist-test-slave-5w2x krb5kdc[17230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851457, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:17.013921 17243 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:44:17.015401 17243 webserver.cc:460] Webserver started at http://127.16.79.126:41979/ using document root <none> and password file <none>
I20260619 06:44:17.016084 17243 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:17.016260 17243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:17.016584 17243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:17.019411 17243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "ec7e9cdf462a4a3e9fe641d91690b8c0"
format_stamp: "Formatted at 2026-06-19 06:44:17 on dist-test-slave-5w2x"
server_key: "2bfa9b21c467e6f94934e0741d0acb82"
server_key_iv: "424dba64a7cc3ee7aa29a7c88610227d"
server_key_version: "encryptionkey@0"
I20260619 06:44:17.020215 17243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "ec7e9cdf462a4a3e9fe641d91690b8c0"
format_stamp: "Formatted at 2026-06-19 06:44:17 on dist-test-slave-5w2x"
server_key: "2bfa9b21c467e6f94934e0741d0acb82"
server_key_iv: "424dba64a7cc3ee7aa29a7c88610227d"
server_key_version: "encryptionkey@0"
I20260619 06:44:17.025174 17243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260619 06:44:17.028535 17258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:17.029873 17243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260619 06:44:17.030071 17243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "ec7e9cdf462a4a3e9fe641d91690b8c0"
format_stamp: "Formatted at 2026-06-19 06:44:17 on dist-test-slave-5w2x"
server_key: "2bfa9b21c467e6f94934e0741d0acb82"
server_key_iv: "424dba64a7cc3ee7aa29a7c88610227d"
server_key_version: "encryptionkey@0"
I20260619 06:44:17.030273 17243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:17.046077 17243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:17.050367 17243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:17.050659 17243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:17.068739 17243 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:44761
I20260619 06:44:17.068771 17309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:44761 every 8 connection(s)
I20260619 06:44:17.070075 17243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:44:17.073719 17310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:17.079123 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 17243
I20260619 06:44:17.079363 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:44:17.080165 16701 external_mini_cluster.cc:1428] Setting key 01d0b10bee4dccd3631eca5e3720e1a8
I20260619 06:44:17.086262 17310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0: Bootstrap starting.
I20260619 06:44:17.089371 17310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:17.090342 17310 log.cc:824] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:17.093240 17310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0: No bootstrap required, opened a new log
Jun 19 06:44:17 dist-test-slave-5w2x krb5kdc[17230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:17.102305 17310 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7e9cdf462a4a3e9fe641d91690b8c0" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44761 } }
I20260619 06:44:17.102785 17310 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:17.102944 17310 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec7e9cdf462a4a3e9fe641d91690b8c0, State: Initialized, Role: FOLLOWER
I20260619 06:44:17.103605 17310 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7e9cdf462a4a3e9fe641d91690b8c0" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44761 } }
I20260619 06:44:17.103905 17310 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:44:17.104038 17310 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:44:17.104257 17310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:17.107363 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:17.082237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43624 (local address 127.16.79.126:44761)
0619 06:44:17.086220 (+ 3983us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:17.086231 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:17.086273 (+ 42us) server_negotiation.cc:407] Connection header received
0619 06:44:17.088018 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:17.088047 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:17.088474 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:17.089086 (+ 612us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:17.090608 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:17.092419 (+ 1811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:17.093758 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:17.094700 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:17.094892 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:17.098838 (+ 3946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:17.098848 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:17.098871 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:17.098923 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:17.102753 (+ 3830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:17.103441 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:17.103449 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:17.103456 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:17.103656 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:17.104102 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:17.104109 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:17.104114 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:17.104878 (+ 764us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:17.105180 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:17.105596 (+ 416us) server_negotiation.cc:299] Negotiation successful
0619 06:44:17.106024 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3726,"thread_start_us":3498,"threads_started":1}
I20260619 06:44:17.107483 17310 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7e9cdf462a4a3e9fe641d91690b8c0" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44761 } }
I20260619 06:44:17.108070 17310 leader_election.cc:302] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec7e9cdf462a4a3e9fe641d91690b8c0; no voters:
I20260619 06:44:17.108711 17310 leader_election.cc:288] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:44:17.108834 17315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:17.110034 17315 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [term 1 LEADER]: Becoming Leader. State: Replica: ec7e9cdf462a4a3e9fe641d91690b8c0, State: Running, Role: LEADER
I20260619 06:44:17.110675 17315 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7e9cdf462a4a3e9fe641d91690b8c0" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44761 } }
I20260619 06:44:17.110909 17310 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:44:17.113591 17316 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec7e9cdf462a4a3e9fe641d91690b8c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7e9cdf462a4a3e9fe641d91690b8c0" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44761 } } }
I20260619 06:44:17.113744 17317 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec7e9cdf462a4a3e9fe641d91690b8c0. Latest consensus state: current_term: 1 leader_uuid: "ec7e9cdf462a4a3e9fe641d91690b8c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec7e9cdf462a4a3e9fe641d91690b8c0" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44761 } } }
I20260619 06:44:17.114075 17317 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:17.114075 17316 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0 [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:17.117182 17321 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:44:17.121456 17321 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:44:17.131076 17321 catalog_manager.cc:1347] Generated new cluster ID: b7e215427ee34799b968e67ed588d78d
I20260619 06:44:17.131191 17321 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:44:17.141301 17321 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:44:17.142634 17321 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:44:17.149261 17321 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec7e9cdf462a4a3e9fe641d91690b8c0: Generated new TSK 0
I20260619 06:44:17.150224 17321 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:44:17.187813 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:44761
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:17.507035 17336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:17.507329 17336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:17.507503 17336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:17.514900 17336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:17.515005 17336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:17.515202 17336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:44:17.525282 17336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:44761
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:17.526572 17336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:17.527930 17336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:17.541914 17341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:17.541879 17344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:17.541981 17342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:17.542703 17336 server_base.cc:1034] running on GCE node
I20260619 06:44:17.543416 17336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:17.544458 17336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:17.545710 17336 hybrid_clock.cc:648] HybridClock initialized: now 1781851457545669 us; error 56 us; skew 500 ppm
I20260619 06:44:17.546062 17336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:17 dist-test-slave-5w2x krb5kdc[17230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851457, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:17.549306 17336 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:44:17.550733 17336 webserver.cc:460] Webserver started at http://127.16.79.65:40811/ using document root <none> and password file <none>
I20260619 06:44:17.551409 17336 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:17.551503 17336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:17.551752 17336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:17.554629 17336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "99e9bfab50724fc1814754b04b7d147a"
format_stamp: "Formatted at 2026-06-19 06:44:17 on dist-test-slave-5w2x"
server_key: "75ec31925c7ac485f1412c5782192fb6"
server_key_iv: "e09c7f0cf2838117fb8b287111b4a616"
server_key_version: "encryptionkey@0"
I20260619 06:44:17.555341 17336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "99e9bfab50724fc1814754b04b7d147a"
format_stamp: "Formatted at 2026-06-19 06:44:17 on dist-test-slave-5w2x"
server_key: "75ec31925c7ac485f1412c5782192fb6"
server_key_iv: "e09c7f0cf2838117fb8b287111b4a616"
server_key_version: "encryptionkey@0"
I20260619 06:44:17.560441 17336 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:44:17.563865 17351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:17.565348 17336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:44:17.565513 17336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "99e9bfab50724fc1814754b04b7d147a"
format_stamp: "Formatted at 2026-06-19 06:44:17 on dist-test-slave-5w2x"
server_key: "75ec31925c7ac485f1412c5782192fb6"
server_key_iv: "e09c7f0cf2838117fb8b287111b4a616"
server_key_version: "encryptionkey@0"
I20260619 06:44:17.565711 17336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:17.579456 17336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:17.583511 17336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:17.583801 17336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:17.584784 17336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:17.586030 17336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:17.586112 17336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:17.586272 17336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:17.586337 17336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:17.619863 17336 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:41413
I20260619 06:44:17.619913 17463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:41413 every 8 connection(s)
I20260619 06:44:17.621184 17336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:44:17.624104 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 17336
I20260619 06:44:17.624341 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:44:17.625183 16701 external_mini_cluster.cc:1428] Setting key 5fc61bb87650eeafdb6b067da833059c
Jun 19 06:44:17 dist-test-slave-5w2x krb5kdc[17230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851457, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
I20260619 06:44:17.643016 17466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:17.623779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:60923 (local address 127.16.79.126:44761)
0619 06:44:17.624277 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:17.624282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:17.625826 (+ 1544us) server_negotiation.cc:407] Connection header received
0619 06:44:17.627955 (+ 2129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:17.627961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:17.628061 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:17.628210 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:17.629794 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:17.630968 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:17.632836 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:17.633341 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:17.633489 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:17.637150 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:17.637157 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:17.637161 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:17.637202 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:17.639780 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:17.640599 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:17.640605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:17.640608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:17.640731 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:17.641171 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:17.641176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:17.641180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:17.641456 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:17.641620 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:17.642409 (+ 789us) server_negotiation.cc:299] Negotiation successful
0619 06:44:17.642631 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":256,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
I20260619 06:44:17.644184 17467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:17.624539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:44761 (local address 127.16.79.65:60923)
0619 06:44:17.625435 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:17.625495 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:17.627675 (+ 2180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:17.628441 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:17.628451 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:17.628957 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:17.629554 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:17.629571 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:17.631181 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:17.631186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:17.632671 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:17.632684 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:17.633613 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:17.633623 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:17.633876 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:17.634634 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:17.634659 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:17.636908 (+ 2249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:17.639930 (+ 3022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:17.639939 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:17.639958 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:17.640469 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:17.640842 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:17.640848 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:17.640852 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:17.641048 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:17.641595 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:17.641603 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:17.641976 (+ 373us) client_negotiation.cc:769] Sending connection context
0619 06:44:17.642315 (+ 339us) client_negotiation.cc:240] Negotiation successful
0619 06:44:17.642895 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":325,"threads_started":1}
I20260619 06:44:17.646181 17464 heartbeater.cc:347] Connected to a master server at 127.16.79.126:44761
I20260619 06:44:17.646507 17464 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:17.647244 17464 heartbeater.cc:511] Master 127.16.79.126:44761 requested a full tablet report, sending...
I20260619 06:44:17.649744 17275 ts_manager.cc:194] Registered new tserver with Master: 99e9bfab50724fc1814754b04b7d147a (127.16.79.65:41413)
I20260619 06:44:17.651971 17275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:60923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:44:17.665568 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:44761
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:17.980911 17470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:17.981202 17470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:17.981298 17470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:17.988597 17470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:17.988705 17470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:17.988817 17470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:44:17.998184 17470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:44761
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:17.999370 17470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:18.000638 17470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:18.013448 17475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:18.013396 17476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:18.013425 17478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:18.014248 17470 server_base.cc:1034] running on GCE node
I20260619 06:44:18.014899 17470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:18.015784 17470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:18.016996 17470 hybrid_clock.cc:648] HybridClock initialized: now 1781851458016927 us; error 49 us; skew 500 ppm
I20260619 06:44:18.017324 17470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:18 dist-test-slave-5w2x krb5kdc[17230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851458, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:18.020390 17470 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:44:18.021768 17470 webserver.cc:460] Webserver started at http://127.16.79.66:34151/ using document root <none> and password file <none>
I20260619 06:44:18.022428 17470 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:18.022563 17470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:18.022866 17470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:18.025681 17470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "c1cf60ec25384eb3b9bed6067b8a9325"
format_stamp: "Formatted at 2026-06-19 06:44:18 on dist-test-slave-5w2x"
server_key: "dd9538c3370585b5d788e3e2864e5ebc"
server_key_iv: "cac9b97482607a8a8bbe15a9ee6ceb14"
server_key_version: "encryptionkey@0"
I20260619 06:44:18.026440 17470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "c1cf60ec25384eb3b9bed6067b8a9325"
format_stamp: "Formatted at 2026-06-19 06:44:18 on dist-test-slave-5w2x"
server_key: "dd9538c3370585b5d788e3e2864e5ebc"
server_key_iv: "cac9b97482607a8a8bbe15a9ee6ceb14"
server_key_version: "encryptionkey@0"
I20260619 06:44:18.031298 17470 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260619 06:44:18.034778 17485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:18.036105 17470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:44:18.036353 17470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "c1cf60ec25384eb3b9bed6067b8a9325"
format_stamp: "Formatted at 2026-06-19 06:44:18 on dist-test-slave-5w2x"
server_key: "dd9538c3370585b5d788e3e2864e5ebc"
server_key_iv: "cac9b97482607a8a8bbe15a9ee6ceb14"
server_key_version: "encryptionkey@0"
I20260619 06:44:18.036588 17470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:18.047931 17470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:18.052121 17470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:18.052476 17470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:18.053434 17470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:18.054793 17470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:18.054894 17470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:18.055032 17470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:18.055104 17470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:18.090909 17470 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:45921
I20260619 06:44:18.090943 17597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:45921 every 8 connection(s)
I20260619 06:44:18.092293 17470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:44:18.101713 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 17470
I20260619 06:44:18.101907 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:44:18.102499 16701 external_mini_cluster.cc:1428] Setting key f7bf12e91d2faf9ffda2c9c8ac647496
Jun 19 06:44:18 dist-test-slave-5w2x krb5kdc[17230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851458, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:18.113744 17466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.094979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:54893 (local address 127.16.79.126:44761)
0619 06:44:18.095176 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:18.095181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:18.096302 (+ 1121us) server_negotiation.cc:407] Connection header received
0619 06:44:18.098288 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:18.098294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:18.098389 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:18.098507 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:18.100100 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.100954 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.102877 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.103505 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.103660 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.107360 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:18.107366 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:18.107369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:18.107404 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:18.110218 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:18.111132 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:18.111137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:18.111139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:18.111303 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:18.111772 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:18.111778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:18.111782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:18.112193 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:18.112355 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:18.113128 (+ 773us) server_negotiation.cc:299] Negotiation successful
0619 06:44:18.113369 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 06:44:18.114967 17600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.095386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:44761 (local address 127.16.79.66:54893)
0619 06:44:18.096087 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:18.096183 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:18.097994 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:18.098770 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:18.098779 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:18.099320 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:18.099917 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:18.099934 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.101111 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.101119 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:18.102708 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:18.102731 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.103754 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.103765 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:18.104023 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.104821 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:18.104846 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:18.107120 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:18.110373 (+ 3253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:18.110383 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:18.110402 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:18.110978 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:18.111420 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:18.111427 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:18.111431 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:18.111635 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:18.112344 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:18.112353 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:18.112727 (+ 374us) client_negotiation.cc:769] Sending connection context
0619 06:44:18.113019 (+ 292us) client_negotiation.cc:240] Negotiation successful
0619 06:44:18.113621 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":271,"threads_started":1}
I20260619 06:44:18.116654 17598 heartbeater.cc:347] Connected to a master server at 127.16.79.126:44761
I20260619 06:44:18.116990 17598 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
I20260619 06:44:18.117697 17598 heartbeater.cc:511] Master 127.16.79.126:44761 requested a full tablet report, sending...
I20260619 06:44:18.119259 17275 ts_manager.cc:194] Registered new tserver with Master: c1cf60ec25384eb3b9bed6067b8a9325 (127.16.79.66:45921)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
I20260619 06:44:18.120339 17275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:54893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:44:18.141474 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:44761
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:36603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:18.453543 17603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:18.453828 17603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:18.454010 17603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:18.461099 17603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:18.461203 17603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:18.461412 17603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:44:18.471199 17603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:36603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:44761
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:18.472519 17603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:18.473807 17603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:18.486439 17608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:18.486518 17609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:18.486477 17611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:18.487502 17603 server_base.cc:1034] running on GCE node
I20260619 06:44:18.488052 17603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:18.488994 17603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:18.490213 17603 hybrid_clock.cc:648] HybridClock initialized: now 1781851458490156 us; error 54 us; skew 500 ppm
I20260619 06:44:18.490516 17603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:18 dist-test-slave-5w2x krb5kdc[17230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851458, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:18.493496 17603 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:44:18.494851 17603 webserver.cc:460] Webserver started at http://127.16.79.67:37499/ using document root <none> and password file <none>
I20260619 06:44:18.495468 17603 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:18.495560 17603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:18.495812 17603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:18.498569 17603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "125bd3a41c3f4472b4f6bcd63979296f"
format_stamp: "Formatted at 2026-06-19 06:44:18 on dist-test-slave-5w2x"
server_key: "a90cc145fb49ae933bc5d3d2db5967d5"
server_key_iv: "6df9d5931f0bce423991e44754d3d77e"
server_key_version: "encryptionkey@0"
I20260619 06:44:18.499254 17603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "125bd3a41c3f4472b4f6bcd63979296f"
format_stamp: "Formatted at 2026-06-19 06:44:18 on dist-test-slave-5w2x"
server_key: "a90cc145fb49ae933bc5d3d2db5967d5"
server_key_iv: "6df9d5931f0bce423991e44754d3d77e"
server_key_version: "encryptionkey@0"
I20260619 06:44:18.504072 17603 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:44:18.507477 17618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:18.508888 17603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 06:44:18.509017 17603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "125bd3a41c3f4472b4f6bcd63979296f"
format_stamp: "Formatted at 2026-06-19 06:44:18 on dist-test-slave-5w2x"
server_key: "a90cc145fb49ae933bc5d3d2db5967d5"
server_key_iv: "6df9d5931f0bce423991e44754d3d77e"
server_key_version: "encryptionkey@0"
I20260619 06:44:18.509178 17603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:18.525130 17603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:18.529174 17603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:18.529402 17603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:18.530252 17603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:18.531440 17603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:18.531510 17603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:18.531591 17603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:18.531642 17603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:18.563928 17603 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:36947
I20260619 06:44:18.563972 17730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:36947 every 8 connection(s)
I20260619 06:44:18.565431 17603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:44:18.567277 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 17603
I20260619 06:44:18.567482 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:44:18.568002 16701 external_mini_cluster.cc:1428] Setting key 8326eb6fd16384b911eff9f8f1734dff
Jun 19 06:44:18 dist-test-slave-5w2x krb5kdc[17230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851458, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:18.585471 17466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.568030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:38849 (local address 127.16.79.126:44761)
0619 06:44:18.568246 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:18.568252 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:18.569660 (+ 1408us) server_negotiation.cc:407] Connection header received
0619 06:44:18.571609 (+ 1949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:18.571613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:18.571678 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:18.571758 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:18.573278 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.574018 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.575605 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.576011 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.576117 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.579564 (+ 3447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:18.579569 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:18.579572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:18.579601 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:18.582348 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:18.583211 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:18.583216 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:18.583219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:18.583347 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:18.583765 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:18.583768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:18.583771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:18.584021 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:18.584194 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:18.585054 (+ 860us) server_negotiation.cc:299] Negotiation successful
0619 06:44:18.585237 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 06:44:18.586575 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.568607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:44761 (local address 127.16.79.67:38849)
0619 06:44:18.569490 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:18.569541 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:18.571364 (+ 1823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:18.571967 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:18.571976 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:18.572542 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:18.573118 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:18.573136 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.574206 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.574210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:18.575462 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:18.575473 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.576304 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.576310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:18.576508 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.577210 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:18.577232 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:18.579341 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:18.582474 (+ 3133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:18.582480 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:18.582494 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:18.583084 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:18.583469 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:18.583472 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:18.583475 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:18.583662 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:18.584131 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:18.584175 (+ 44us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:18.584679 (+ 504us) client_negotiation.cc:769] Sending connection context
0619 06:44:18.584914 (+ 235us) client_negotiation.cc:240] Negotiation successful
0619 06:44:18.585398 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":406,"threads_started":1}
I20260619 06:44:18.587930 17731 heartbeater.cc:347] Connected to a master server at 127.16.79.126:44761
I20260619 06:44:18.588253 17731 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:18.588949 17731 heartbeater.cc:511] Master 127.16.79.126:44761 requested a full tablet report, sending...
I20260619 06:44:18.590261 17275 ts_manager.cc:194] Registered new tserver with Master: 125bd3a41c3f4472b4f6bcd63979296f (127.16.79.67:36947)
I20260619 06:44:18.590997 17275 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:38849
I20260619 06:44:18.595556 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:44:18.613776 17466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.601814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43628 (local address 127.16.79.126:44761)
0619 06:44:18.601981 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:18.601985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:18.602502 (+ 517us) server_negotiation.cc:407] Connection header received
0619 06:44:18.602638 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:18.602641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:18.602700 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:18.602786 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:18.604211 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.604930 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.606138 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.606538 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.606625 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.608979 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:18.608984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:18.608987 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:18.609017 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:18.611365 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:18.611985 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:18.611988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:18.611990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:18.612220 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:18.612536 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:18.612540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:18.612541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:18.612856 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:18.613028 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:18.613361 (+ 333us) server_negotiation.cc:299] Negotiation successful
0619 06:44:18.613509 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:44:18.620277 17275 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:44:18.628979 17275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:44:18.655010 17464 heartbeater.cc:503] Master 127.16.79.126:44761 was elected leader, sending a full tablet report...
I20260619 06:44:18.662388 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.640953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:36947 (local address 127.0.0.1:49962)
0619 06:44:18.654151 (+ 13198us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:18.654170 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:18.654306 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:18.655313 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:18.655319 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:18.655346 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:18.655838 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:18.655850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.657788 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.657793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:18.660004 (+ 2211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:18.660017 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.661282 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.661288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:18.661466 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.661526 (+ 60us) client_negotiation.cc:769] Sending connection context
0619 06:44:18.661684 (+ 158us) client_negotiation.cc:240] Negotiation successful
0619 06:44:18.661877 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13042,"mutex_wait_us":24,"spinlock_wait_cycles":1624320,"thread_start_us":12883,"threads_started":1}
I20260619 06:44:18.662536 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.641080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49962 (local address 127.16.79.67:36947)
0619 06:44:18.653793 (+ 12713us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:18.653805 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:18.654438 (+ 633us) server_negotiation.cc:407] Connection header received
0619 06:44:18.654557 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:18.654566 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:18.654841 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:18.655043 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:18.656058 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.657595 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.660186 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.661074 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.661242 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.661474 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:18.662010 (+ 536us) server_negotiation.cc:299] Negotiation successful
0619 06:44:18.662172 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12539,"thread_start_us":12390,"threads_started":1}
I20260619 06:44:18.663196 17745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.640531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:45921 (local address 127.0.0.1:42562)
0619 06:44:18.651552 (+ 11021us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:18.651598 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:18.651798 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:18.656405 (+ 4607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:18.656409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:18.656433 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:18.656936 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:18.656947 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.658700 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.658704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:18.660320 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:18.660335 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.662361 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.662367 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:18.662557 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.662603 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 06:44:18.662687 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 06:44:18.662813 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10771,"mutex_wait_us":24,"thread_start_us":10630,"threads_started":1}
I20260619 06:44:18.663210 17746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.641439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39068 (local address 127.16.79.65:41413)
0619 06:44:18.653346 (+ 11907us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:18.653357 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:18.654415 (+ 1058us) server_negotiation.cc:407] Connection header received
0619 06:44:18.654555 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:18.654567 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:18.654844 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:18.655066 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:18.655798 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.657172 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.660033 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.661527 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.661679 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.661887 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:18.662685 (+ 798us) server_negotiation.cc:299] Negotiation successful
0619 06:44:18.662854 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11737,"thread_start_us":11563,"threads_started":1}
I20260619 06:44:18.662577 17747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.641634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:41413 (local address 127.0.0.1:39068)
0619 06:44:18.654271 (+ 12637us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:18.654286 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:18.654392 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:18.655074 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:18.655083 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:18.655119 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:18.655570 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:18.655588 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.657307 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.657312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:18.659867 (+ 2555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:18.659881 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.661743 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.661748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:18.661901 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.661928 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 06:44:18.662017 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 06:44:18.662131 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12493,"mutex_wait_us":35,"thread_start_us":12418,"threads_started":1}
I20260619 06:44:18.663403 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.640724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42562 (local address 127.16.79.66:45921)
0619 06:44:18.655582 (+ 14858us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:18.655590 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:18.655617 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:44:18.655712 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:18.655722 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:18.655950 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:18.656158 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:18.657103 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.658533 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.660501 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.661888 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.662020 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.662189 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:18.662895 (+ 706us) server_negotiation.cc:299] Negotiation successful
0619 06:44:18.663067 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13980,"thread_start_us":13856,"threads_started":1}
I20260619 06:44:18.666596 17666 tablet_service.cc:1467] Processing CreateTablet for tablet 17e7b4383b874a1f980d02f78f3c7247 (DEFAULT_TABLE table=test-table [id=07127e46bfb241978bf809acda87bda8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:18.667202 17533 tablet_service.cc:1467] Processing CreateTablet for tablet 17e7b4383b874a1f980d02f78f3c7247 (DEFAULT_TABLE table=test-table [id=07127e46bfb241978bf809acda87bda8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:18.667191 17399 tablet_service.cc:1467] Processing CreateTablet for tablet 17e7b4383b874a1f980d02f78f3c7247 (DEFAULT_TABLE table=test-table [id=07127e46bfb241978bf809acda87bda8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:18.667912 17666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17e7b4383b874a1f980d02f78f3c7247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:18.668511 17399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17e7b4383b874a1f980d02f78f3c7247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:18.668503 17533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17e7b4383b874a1f980d02f78f3c7247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:18.678930 17749 tablet_bootstrap.cc:492] T 17e7b4383b874a1f980d02f78f3c7247 P 125bd3a41c3f4472b4f6bcd63979296f: Bootstrap starting.
I20260619 06:44:18.682678 17750 tablet_bootstrap.cc:492] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a: Bootstrap starting.
I20260619 06:44:18.683451 17751 tablet_bootstrap.cc:492] T 17e7b4383b874a1f980d02f78f3c7247 P c1cf60ec25384eb3b9bed6067b8a9325: Bootstrap starting.
I20260619 06:44:18.684578 17749 tablet_bootstrap.cc:654] T 17e7b4383b874a1f980d02f78f3c7247 P 125bd3a41c3f4472b4f6bcd63979296f: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:18.685904 17749 log.cc:824] T 17e7b4383b874a1f980d02f78f3c7247 P 125bd3a41c3f4472b4f6bcd63979296f: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:18.686367 17750 tablet_bootstrap.cc:654] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:18.687057 17751 tablet_bootstrap.cc:654] T 17e7b4383b874a1f980d02f78f3c7247 P c1cf60ec25384eb3b9bed6067b8a9325: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:18.687407 17750 log.cc:824] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:18.688512 17751 log.cc:824] T 17e7b4383b874a1f980d02f78f3c7247 P c1cf60ec25384eb3b9bed6067b8a9325: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:18.690611 17750 tablet_bootstrap.cc:492] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a: No bootstrap required, opened a new log
I20260619 06:44:18.690719 17749 tablet_bootstrap.cc:492] T 17e7b4383b874a1f980d02f78f3c7247 P 125bd3a41c3f4472b4f6bcd63979296f: No bootstrap required, opened a new log
I20260619 06:44:18.691010 17749 ts_tablet_manager.cc:1397] T 17e7b4383b874a1f980d02f78f3c7247 P 125bd3a41c3f4472b4f6bcd63979296f: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260619 06:44:18.691486 17750 ts_tablet_manager.cc:1397] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260619 06:44:18.691648 17751 tablet_bootstrap.cc:492] T 17e7b4383b874a1f980d02f78f3c7247 P c1cf60ec25384eb3b9bed6067b8a9325: No bootstrap required, opened a new log
I20260619 06:44:18.691908 17751 ts_tablet_manager.cc:1397] T 17e7b4383b874a1f980d02f78f3c7247 P c1cf60ec25384eb3b9bed6067b8a9325: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260619 06:44:18.698565 17749 raft_consensus.cc:348] T 17e7b4383b874a1f980d02f78f3c7247 P 125bd3a41c3f4472b4f6bcd63979296f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cf60ec25384eb3b9bed6067b8a9325" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45921 } } peers { permanent_uuid: "125bd3a41c3f4472b4f6bcd63979296f" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36947 } } peers { permanent_uuid: "99e9bfab50724fc1814754b04b7d147a" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 41413 } }
I20260619 06:44:18.698959 17749 raft_consensus.cc:374] T 17e7b4383b874a1f980d02f78f3c7247 P 125bd3a41c3f4472b4f6bcd63979296f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:18.699067 17749 raft_consensus.cc:729] T 17e7b4383b874a1f980d02f78f3c7247 P 125bd3a41c3f4472b4f6bcd63979296f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125bd3a41c3f4472b4f6bcd63979296f, State: Initialized, Role: FOLLOWER
I20260619 06:44:18.699697 17749 consensus_queue.cc:260] T 17e7b4383b874a1f980d02f78f3c7247 P 125bd3a41c3f4472b4f6bcd63979296f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cf60ec25384eb3b9bed6067b8a9325" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45921 } } peers { permanent_uuid: "125bd3a41c3f4472b4f6bcd63979296f" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36947 } } peers { permanent_uuid: "99e9bfab50724fc1814754b04b7d147a" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 41413 } }
I20260619 06:44:18.700837 17731 heartbeater.cc:503] Master 127.16.79.126:44761 was elected leader, sending a full tablet report...
I20260619 06:44:18.701552 17749 ts_tablet_manager.cc:1428] T 17e7b4383b874a1f980d02f78f3c7247 P 125bd3a41c3f4472b4f6bcd63979296f: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260619 06:44:18.702194 17751 raft_consensus.cc:348] T 17e7b4383b874a1f980d02f78f3c7247 P c1cf60ec25384eb3b9bed6067b8a9325 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cf60ec25384eb3b9bed6067b8a9325" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45921 } } peers { permanent_uuid: "125bd3a41c3f4472b4f6bcd63979296f" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36947 } } peers { permanent_uuid: "99e9bfab50724fc1814754b04b7d147a" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 41413 } }
I20260619 06:44:18.702208 17750 raft_consensus.cc:348] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cf60ec25384eb3b9bed6067b8a9325" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45921 } } peers { permanent_uuid: "125bd3a41c3f4472b4f6bcd63979296f" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36947 } } peers { permanent_uuid: "99e9bfab50724fc1814754b04b7d147a" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 41413 } }
I20260619 06:44:18.702627 17750 raft_consensus.cc:374] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:18.702662 17751 raft_consensus.cc:374] T 17e7b4383b874a1f980d02f78f3c7247 P c1cf60ec25384eb3b9bed6067b8a9325 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:18.702740 17750 raft_consensus.cc:729] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99e9bfab50724fc1814754b04b7d147a, State: Initialized, Role: FOLLOWER
I20260619 06:44:18.702759 17751 raft_consensus.cc:729] T 17e7b4383b874a1f980d02f78f3c7247 P c1cf60ec25384eb3b9bed6067b8a9325 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1cf60ec25384eb3b9bed6067b8a9325, State: Initialized, Role: FOLLOWER
I20260619 06:44:18.703351 17750 consensus_queue.cc:260] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cf60ec25384eb3b9bed6067b8a9325" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45921 } } peers { permanent_uuid: "125bd3a41c3f4472b4f6bcd63979296f" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36947 } } peers { permanent_uuid: "99e9bfab50724fc1814754b04b7d147a" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 41413 } }
I20260619 06:44:18.703351 17751 consensus_queue.cc:260] T 17e7b4383b874a1f980d02f78f3c7247 P c1cf60ec25384eb3b9bed6067b8a9325 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cf60ec25384eb3b9bed6067b8a9325" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45921 } } peers { permanent_uuid: "125bd3a41c3f4472b4f6bcd63979296f" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36947 } } peers { permanent_uuid: "99e9bfab50724fc1814754b04b7d147a" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 41413 } }
I20260619 06:44:18.708860 17598 heartbeater.cc:503] Master 127.16.79.126:44761 was elected leader, sending a full tablet report...
I20260619 06:44:18.709367 17750 ts_tablet_manager.cc:1428] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a: Time spent starting tablet: real 0.018s user 0.004s sys 0.011s
I20260619 06:44:18.710435 17751 ts_tablet_manager.cc:1428] T 17e7b4383b874a1f980d02f78f3c7247 P c1cf60ec25384eb3b9bed6067b8a9325: Time spent starting tablet: real 0.018s user 0.011s sys 0.007s
W20260619 06:44:18.816989 17732 tablet.cc:2369] T 17e7b4383b874a1f980d02f78f3c7247 P 125bd3a41c3f4472b4f6bcd63979296f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:44:18.838949 17757 raft_consensus.cc:482] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:44:18.839226 17757 raft_consensus.cc:504] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cf60ec25384eb3b9bed6067b8a9325" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45921 } } peers { permanent_uuid: "125bd3a41c3f4472b4f6bcd63979296f" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36947 } } peers { permanent_uuid: "99e9bfab50724fc1814754b04b7d147a" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 41413 } }
I20260619 06:44:18.841084 17757 leader_election.cc:288] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1cf60ec25384eb3b9bed6067b8a9325 (127.16.79.66:45921), 125bd3a41c3f4472b4f6bcd63979296f (127.16.79.67:36947)
W20260619 06:44:18.844352 17599 tablet.cc:2369] T 17e7b4383b874a1f980d02f78f3c7247 P c1cf60ec25384eb3b9bed6067b8a9325: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:44:18.847736 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.841410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:55043 (local address 127.16.79.66:45921)
0619 06:44:18.841685 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:18.841695 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:18.842439 (+ 744us) server_negotiation.cc:407] Connection header received
0619 06:44:18.842677 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:18.842681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:18.842739 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:18.842839 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:18.843384 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.844199 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.846084 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.846827 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.846995 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.847071 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:18.847348 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 06:44:18.847473 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:44:18.847868 17758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.841410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:45921 (local address 127.16.79.65:55043)
0619 06:44:18.842324 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:18.842344 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:18.842519 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:18.842855 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:18.842859 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:18.842885 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:18.843244 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:18.843252 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.844327 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.844334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:18.845921 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:18.845932 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.847030 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.847036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:18.847190 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.847214 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 06:44:18.847323 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 06:44:18.847458 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"mutex_wait_us":4,"thread_start_us":486,"threads_started":1}
I20260619 06:44:18.848088 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.841637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:33531 (local address 127.16.79.67:36947)
0619 06:44:18.841866 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:18.841875 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:18.842659 (+ 784us) server_negotiation.cc:407] Connection header received
0619 06:44:18.842860 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:18.842864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:18.842918 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:18.843012 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:18.843568 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.844385 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.846351 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.847209 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.847387 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.847461 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:18.847726 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 06:44:18.847844 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 06:44:18.848312 17759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.841496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:36947 (local address 127.16.79.65:33531)
0619 06:44:18.842545 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:18.842559 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:18.842708 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:18.843028 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:18.843032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:18.843059 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:18.843423 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:18.843430 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.844553 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.844561 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:18.846200 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:18.846211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.847429 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.847435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:18.847573 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.847599 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 06:44:18.847702 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 06:44:18.847827 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":955,"mutex_wait_us":28,"thread_start_us":685,"threads_started":1}
I20260619 06:44:18.849099 17553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17e7b4383b874a1f980d02f78f3c7247" candidate_uuid: "99e9bfab50724fc1814754b04b7d147a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1cf60ec25384eb3b9bed6067b8a9325" is_pre_election: true
I20260619 06:44:18.849256 17686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17e7b4383b874a1f980d02f78f3c7247" candidate_uuid: "99e9bfab50724fc1814754b04b7d147a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125bd3a41c3f4472b4f6bcd63979296f" is_pre_election: true
I20260619 06:44:18.849627 17553 raft_consensus.cc:2454] T 17e7b4383b874a1f980d02f78f3c7247 P c1cf60ec25384eb3b9bed6067b8a9325 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99e9bfab50724fc1814754b04b7d147a in term 0.
I20260619 06:44:18.849737 17686 raft_consensus.cc:2454] T 17e7b4383b874a1f980d02f78f3c7247 P 125bd3a41c3f4472b4f6bcd63979296f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99e9bfab50724fc1814754b04b7d147a in term 0.
I20260619 06:44:18.850319 17354 leader_election.cc:302] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99e9bfab50724fc1814754b04b7d147a, c1cf60ec25384eb3b9bed6067b8a9325; no voters:
I20260619 06:44:18.850728 17757 raft_consensus.cc:2781] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:44:18.850881 17757 raft_consensus.cc:482] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:44:18.850991 17757 raft_consensus.cc:3037] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:18.853190 17757 raft_consensus.cc:504] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cf60ec25384eb3b9bed6067b8a9325" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45921 } } peers { permanent_uuid: "125bd3a41c3f4472b4f6bcd63979296f" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36947 } } peers { permanent_uuid: "99e9bfab50724fc1814754b04b7d147a" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 41413 } }
I20260619 06:44:18.853794 17757 leader_election.cc:288] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [CANDIDATE]: Term 1 election: Requested vote from peers c1cf60ec25384eb3b9bed6067b8a9325 (127.16.79.66:45921), 125bd3a41c3f4472b4f6bcd63979296f (127.16.79.67:36947)
I20260619 06:44:18.854218 17553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17e7b4383b874a1f980d02f78f3c7247" candidate_uuid: "99e9bfab50724fc1814754b04b7d147a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1cf60ec25384eb3b9bed6067b8a9325"
I20260619 06:44:18.854295 17686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17e7b4383b874a1f980d02f78f3c7247" candidate_uuid: "99e9bfab50724fc1814754b04b7d147a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125bd3a41c3f4472b4f6bcd63979296f"
I20260619 06:44:18.854432 17553 raft_consensus.cc:3037] T 17e7b4383b874a1f980d02f78f3c7247 P c1cf60ec25384eb3b9bed6067b8a9325 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:18.854471 17686 raft_consensus.cc:3037] T 17e7b4383b874a1f980d02f78f3c7247 P 125bd3a41c3f4472b4f6bcd63979296f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:18.857620 17686 raft_consensus.cc:2454] T 17e7b4383b874a1f980d02f78f3c7247 P 125bd3a41c3f4472b4f6bcd63979296f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99e9bfab50724fc1814754b04b7d147a in term 1.
I20260619 06:44:18.857625 17553 raft_consensus.cc:2454] T 17e7b4383b874a1f980d02f78f3c7247 P c1cf60ec25384eb3b9bed6067b8a9325 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99e9bfab50724fc1814754b04b7d147a in term 1.
I20260619 06:44:18.858053 17352 leader_election.cc:302] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125bd3a41c3f4472b4f6bcd63979296f, 99e9bfab50724fc1814754b04b7d147a; no voters:
I20260619 06:44:18.858346 17757 raft_consensus.cc:2781] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:18.858778 17757 raft_consensus.cc:686] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [term 1 LEADER]: Becoming Leader. State: Replica: 99e9bfab50724fc1814754b04b7d147a, State: Running, Role: LEADER
I20260619 06:44:18.859289 17757 consensus_queue.cc:237] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cf60ec25384eb3b9bed6067b8a9325" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45921 } } peers { permanent_uuid: "125bd3a41c3f4472b4f6bcd63979296f" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36947 } } peers { permanent_uuid: "99e9bfab50724fc1814754b04b7d147a" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 41413 } }
I20260619 06:44:18.863479 17274 catalog_manager.cc:5495] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a reported cstate change: term changed from 0 to 1, leader changed from <none> to 99e9bfab50724fc1814754b04b7d147a (127.16.79.65). New cstate: current_term: 1 leader_uuid: "99e9bfab50724fc1814754b04b7d147a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1cf60ec25384eb3b9bed6067b8a9325" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "125bd3a41c3f4472b4f6bcd63979296f" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99e9bfab50724fc1814754b04b7d147a" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 41413 } health_report { overall_health: HEALTHY } } }
W20260619 06:44:18.873390 17465 tablet.cc:2369] T 17e7b4383b874a1f980d02f78f3c7247 P 99e9bfab50724fc1814754b04b7d147a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Jun 19 06:44:18 dist-test-slave-5w2x krb5kdc[17230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:44:18.893988 17746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.879691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39070 (local address 127.16.79.65:41413)
0619 06:44:18.879908 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:18.879916 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:18.880451 (+ 535us) server_negotiation.cc:407] Connection header received
0619 06:44:18.880600 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:18.880604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:18.880678 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:18.880759 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:18.882233 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.882906 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.884120 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.884570 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.884663 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.888693 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:18.888706 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:18.888721 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:18.888754 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:18.891545 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:18.892203 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:18.892211 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:18.892213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:18.892343 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:18.892665 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:18.892668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:18.892671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:18.892985 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:18.893203 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:18.893443 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 06:44:18.893621 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Jun 19 06:44:18 dist-test-slave-5w2x krb5kdc[17230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851458, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 06:44:18 dist-test-slave-5w2x krb5kdc[17230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851458, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:44:18.938720 17746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.925166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39080 (local address 127.16.79.65:41413)
0619 06:44:18.925390 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:18.925398 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:18.925715 (+ 317us) server_negotiation.cc:407] Connection header received
0619 06:44:18.925938 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:18.925942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:18.926004 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:18.926104 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:18.927388 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.928111 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.929409 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.929827 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.929960 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.933662 (+ 3702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:18.933667 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:18.933672 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:18.933727 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:18.936190 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:18.936806 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:18.936812 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:18.936815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:18.936938 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:18.937309 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:18.937315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:18.937319 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:18.937591 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:18.937883 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:18.938138 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 06:44:18.938368 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 06:44:18 dist-test-slave-5w2x krb5kdc[17230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 06:44:18 dist-test-slave-5w2x krb5kdc[17230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:44:18.982607 17746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.968123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39092 (local address 127.16.79.65:41413)
0619 06:44:18.968361 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:18.968369 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:18.968691 (+ 322us) server_negotiation.cc:407] Connection header received
0619 06:44:18.968913 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:18.968917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:18.968994 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:18.969102 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:18.970516 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.971251 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.972537 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.972949 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.973080 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.976884 (+ 3804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:18.976889 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:18.976893 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:18.976933 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:18.979742 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:18.980618 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:18.980623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:18.980625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:18.980771 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:18.981176 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:18.981180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:18.981182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:18.981461 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:18.981729 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:18.982030 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 06:44:18.982262 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260619 06:44:18.984396 17379 tablet_service.cc:2908] Rejecting scan request for tablet 17e7b4383b874a1f980d02f78f3c7247: Uninitialized: clean time has not yet been initialized
I20260619 06:44:18.987594 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 17336
I20260619 06:44:18.999859 17746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:18.988458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39102 (local address 127.16.79.65:41413)
0619 06:44:18.988605 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:18.988609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:18.989083 (+ 474us) server_negotiation.cc:407] Connection header received
0619 06:44:18.989319 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:18.989327 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:18.989425 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:18.989541 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:18.990740 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.991410 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.992579 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:18.992964 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:18.993053 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:18.995088 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:18.995093 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:18.995096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:18.995123 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:18.997727 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:18.998366 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:18.998371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:18.998373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:18.998484 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:18.998787 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:18.998790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:18.998792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:18.999019 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:18.999182 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:18.999436 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 06:44:18.999606 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 06:44:19.000564 17459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:19.097308 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 17336
I20260619 06:44:19.114945 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 17470
Jun 19 06:44:19 dist-test-slave-5w2x krb5kdc[17230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:44:19.129419 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:19.115822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42572 (local address 127.16.79.66:45921)
0619 06:44:19.115979 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:19.115983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:19.115999 (+ 16us) server_negotiation.cc:407] Connection header received
0619 06:44:19.116052 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:19.116056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:19.116113 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:19.116253 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:19.117357 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:19.118088 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:19.119353 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:19.119728 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:19.119845 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:19.123979 (+ 4134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:19.123993 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:19.124008 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:19.124043 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:19.126888 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:19.127558 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:19.127565 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:19.127567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:19.127689 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:19.128043 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:19.128046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:19.128048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:19.128482 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:19.128685 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:19.129006 (+ 321us) server_negotiation.cc:299] Negotiation successful
0619 06:44:19.129161 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:44:19.130125 17593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:19.218506 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 17470
I20260619 06:44:19.234982 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 17603
Jun 19 06:44:19 dist-test-slave-5w2x krb5kdc[17230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:44:19.249686 17744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:19.235824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49978 (local address 127.16.79.67:36947)
0619 06:44:19.235998 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:19.236003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:19.236021 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:44:19.236241 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:19.236245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:19.236307 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:19.236397 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:19.237579 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:19.238283 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:19.239574 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:19.239943 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:19.240031 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:19.244081 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:19.244097 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:19.244113 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:19.244233 (+ 120us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:19.247023 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:19.247687 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:19.247693 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:19.247695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:19.247829 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:19.248307 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:19.248314 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:19.248321 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:19.248662 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:19.248869 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:19.249163 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 06:44:19.249336 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:44:19.250373 17726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:19.341456 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 17603
I20260619 06:44:19.357461 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 17243
I20260619 06:44:19.358932 17305 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:19.469626 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 17243
2026-06-19T06:44:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4163 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:44:19 dist-test-slave-5w2x krb5kdc[17779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:44:19 dist-test-slave-5w2x krb5kdc[17779](info): set up 2 sockets
Jun 19 06:44:19 dist-test-slave-5w2x krb5kdc[17779](info): commencing operation
krb5kdc: starting...
W20260619 06:44:20.552064 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:44:20 dist-test-slave-5w2x krb5kdc[17779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:44:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:44:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:44:20.729537 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:44803
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:38089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:44803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:21.035851 17792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:21.036201 17792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:21.036373 17792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:21.043310 17792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:44:21.043454 17792 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 06:44:21.043551 17792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:21.043665 17792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:21.043762 17792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:44:21.043850 17792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:44:21.053426 17792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:38089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:44803
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:44803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:21.054733 17792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:21.056173 17792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:21.067756 17797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:21.067819 17800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:21.067797 17798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:21.068790 17792 server_base.cc:1034] running on GCE node
I20260619 06:44:21.069608 17792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:21.071055 17792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:21.072275 17792 hybrid_clock.cc:648] HybridClock initialized: now 1781851461072206 us; error 48 us; skew 500 ppm
I20260619 06:44:21.072610 17792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:21 dist-test-slave-5w2x krb5kdc[17779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851461, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:21.081723 17792 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:44:21.083232 17792 webserver.cc:460] Webserver started at http://127.16.79.126:41655/ using document root <none> and password file <none>
I20260619 06:44:21.083947 17792 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:21.084092 17792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:21.084434 17792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:21.087282 17792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "266db23748be4361a504cebbd39fb039"
format_stamp: "Formatted at 2026-06-19 06:44:21 on dist-test-slave-5w2x"
server_key: "264beb9ea59e45d8cad5d26179bec41d"
server_key_iv: "224d106a2e1e4088efbb49ca7ed51e3b"
server_key_version: "encryptionkey@0"
I20260619 06:44:21.087991 17792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "266db23748be4361a504cebbd39fb039"
format_stamp: "Formatted at 2026-06-19 06:44:21 on dist-test-slave-5w2x"
server_key: "264beb9ea59e45d8cad5d26179bec41d"
server_key_iv: "224d106a2e1e4088efbb49ca7ed51e3b"
server_key_version: "encryptionkey@0"
I20260619 06:44:21.093183 17792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 06:44:21.096688 17807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:21.098428 17792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260619 06:44:21.098636 17792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "266db23748be4361a504cebbd39fb039"
format_stamp: "Formatted at 2026-06-19 06:44:21 on dist-test-slave-5w2x"
server_key: "264beb9ea59e45d8cad5d26179bec41d"
server_key_iv: "224d106a2e1e4088efbb49ca7ed51e3b"
server_key_version: "encryptionkey@0"
I20260619 06:44:21.098927 17792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:21.120560 17792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:21.124640 17792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:21.124938 17792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:21.146173 17792 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:44803
I20260619 06:44:21.146193 17868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:44803 every 8 connection(s)
I20260619 06:44:21.147495 17792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:44:21.151178 17869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:21.155658 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 17792
I20260619 06:44:21.155897 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:44:21.156771 16701 external_mini_cluster.cc:1428] Setting key 0c61c1b48fb46ff2e0fff84b5394ee37
I20260619 06:44:21.162717 17869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039: Bootstrap starting.
I20260619 06:44:21.166479 17869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:21.167618 17869 log.cc:824] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:21.170331 17869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039: No bootstrap required, opened a new log
Jun 19 06:44:21 dist-test-slave-5w2x krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:21.180431 17872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:21.158819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34214 (local address 127.16.79.126:44803)
0619 06:44:21.159632 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:21.159643 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:21.159682 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:44:21.161509 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:21.161531 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:21.161931 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:21.162269 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:21.163958 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:21.165352 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:21.167078 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:21.167830 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:21.167959 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:21.172009 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:21.172019 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:21.172047 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:21.172095 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:21.175976 (+ 3881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:21.176677 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:21.176686 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:21.176695 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:21.176901 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:21.177231 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:21.177236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:21.177239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:21.177941 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:21.178219 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:21.178536 (+ 317us) server_negotiation.cc:299] Negotiation successful
0619 06:44:21.179001 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":385,"threads_started":1}
I20260619 06:44:21.180953 17869 raft_consensus.cc:348] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "266db23748be4361a504cebbd39fb039" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44803 } }
I20260619 06:44:21.181327 17869 raft_consensus.cc:374] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:21.181440 17869 raft_consensus.cc:729] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 266db23748be4361a504cebbd39fb039, State: Initialized, Role: FOLLOWER
I20260619 06:44:21.182085 17869 consensus_queue.cc:260] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "266db23748be4361a504cebbd39fb039" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44803 } }
I20260619 06:44:21.182379 17869 raft_consensus.cc:388] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:44:21.182513 17869 raft_consensus.cc:482] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:44:21.182686 17869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:21.185065 17869 raft_consensus.cc:504] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "266db23748be4361a504cebbd39fb039" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44803 } }
I20260619 06:44:21.185488 17869 leader_election.cc:302] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 266db23748be4361a504cebbd39fb039; no voters:
I20260619 06:44:21.186020 17869 leader_election.cc:288] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:44:21.186158 17874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:21.186893 17874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [term 1 LEADER]: Becoming Leader. State: Replica: 266db23748be4361a504cebbd39fb039, State: Running, Role: LEADER
I20260619 06:44:21.187453 17874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "266db23748be4361a504cebbd39fb039" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44803 } }
I20260619 06:44:21.188097 17869 sys_catalog.cc:564] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:44:21.190260 17875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "266db23748be4361a504cebbd39fb039" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "266db23748be4361a504cebbd39fb039" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44803 } } }
I20260619 06:44:21.190366 17876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 266db23748be4361a504cebbd39fb039. Latest consensus state: current_term: 1 leader_uuid: "266db23748be4361a504cebbd39fb039" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "266db23748be4361a504cebbd39fb039" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44803 } } }
I20260619 06:44:21.190688 17875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:21.190754 17876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039 [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:21.195300 17882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:44:21.200292 17882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:44:21.209561 17882 catalog_manager.cc:1347] Generated new cluster ID: 16568f133d8d40e48608dd5ea3f55d0b
I20260619 06:44:21.209685 17882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:44:21.227025 17882 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:44:21.228348 17882 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:44:21.234596 17882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039: Generated new TSK 0
I20260619 06:44:21.235394 17882 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:44:21.291464 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:44803
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:38089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:21.627250 17895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:21.627532 17895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:21.627687 17895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:21.635023 17895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:21.635131 17895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:21.635280 17895 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:44:21.635408 17895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:44:21.644829 17895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:38089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.79.126:44803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:21.646111 17895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:21.647435 17895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:21.659898 17900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:21.659929 17901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:21.659898 17903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:21.660964 17895 server_base.cc:1034] running on GCE node
I20260619 06:44:21.661468 17895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:21.662290 17895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:21.663532 17895 hybrid_clock.cc:648] HybridClock initialized: now 1781851461663457 us; error 54 us; skew 500 ppm
I20260619 06:44:21.663836 17895 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:21 dist-test-slave-5w2x krb5kdc[17779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851461, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:21.667024 17895 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:44:21.668604 17895 webserver.cc:460] Webserver started at http://127.16.79.65:39503/ using document root <none> and password file <none>
I20260619 06:44:21.669325 17895 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:21.669456 17895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:21.669780 17895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:21.672684 17895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "a091f347648c490abd6a8c2a7d0da576"
format_stamp: "Formatted at 2026-06-19 06:44:21 on dist-test-slave-5w2x"
server_key: "6b7ba70cb39bbc3aab3a932aafeca6b3"
server_key_iv: "81298ca60dddb0905c56ff069ab04a31"
server_key_version: "encryptionkey@0"
I20260619 06:44:21.673466 17895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "a091f347648c490abd6a8c2a7d0da576"
format_stamp: "Formatted at 2026-06-19 06:44:21 on dist-test-slave-5w2x"
server_key: "6b7ba70cb39bbc3aab3a932aafeca6b3"
server_key_iv: "81298ca60dddb0905c56ff069ab04a31"
server_key_version: "encryptionkey@0"
I20260619 06:44:21.678474 17895 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:44:21.681990 17910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:21.683287 17895 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:44:21.683462 17895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "a091f347648c490abd6a8c2a7d0da576"
format_stamp: "Formatted at 2026-06-19 06:44:21 on dist-test-slave-5w2x"
server_key: "6b7ba70cb39bbc3aab3a932aafeca6b3"
server_key_iv: "81298ca60dddb0905c56ff069ab04a31"
server_key_version: "encryptionkey@0"
I20260619 06:44:21.683689 17895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:21.711040 17895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:21.715286 17895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:21.715569 17895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:21.718179 17895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:21.718308 17895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:21.718477 17895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:21.718559 17895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 06:44:21 dist-test-slave-5w2x krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851461, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:21.742391 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:21.718954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:52987 (local address 127.16.79.126:44803)
0619 06:44:21.719451 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:21.719462 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:21.721408 (+ 1946us) server_negotiation.cc:407] Connection header received
0619 06:44:21.723461 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:21.723467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:21.723562 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:21.723682 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:21.726722 (+ 3040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:21.727714 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:21.730372 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:21.730941 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:21.731083 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:21.736276 (+ 5193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:21.736281 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:21.736285 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:21.736317 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:21.738770 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:21.739502 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:21.739506 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:21.739509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:21.739624 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:21.740363 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:21.740371 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:21.740376 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:21.740723 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:21.740895 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:21.741782 (+ 887us) server_negotiation.cc:299] Negotiation successful
0619 06:44:21.742013 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":251,"threads_started":1}
I20260619 06:44:21.744027 17920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:21.720246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:44803 (local address 127.16.79.65:52987)
0619 06:44:21.721208 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:21.721262 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:21.723150 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:21.724044 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:21.724067 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:21.725245 (+ 1178us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:21.726475 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:21.726516 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:21.727916 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:21.727920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:21.730203 (+ 2283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:21.730225 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:21.731205 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:21.731222 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:21.731538 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:21.732445 (+ 907us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:21.732482 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:21.735998 (+ 3516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:21.738918 (+ 2920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:21.738926 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:21.738944 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:21.739370 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:21.739793 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:21.739802 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:21.739808 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:21.740119 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:21.740883 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:21.740891 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:21.741265 (+ 374us) client_negotiation.cc:769] Sending connection context
0619 06:44:21.741683 (+ 418us) client_negotiation.cc:240] Negotiation successful
0619 06:44:21.742282 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":390,"threads_started":1}
I20260619 06:44:21.765601 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:21.750711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:43893 (local address 127.16.79.126:44803)
0619 06:44:21.750872 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:21.750877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:21.751704 (+ 827us) server_negotiation.cc:407] Connection header received
0619 06:44:21.751908 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:21.751920 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:21.751979 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:21.752085 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:21.753346 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:21.754808 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:21.756824 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:21.757376 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:21.757525 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:21.759464 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:21.759469 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:21.759472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:21.759500 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:21.762652 (+ 3152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:21.763554 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:21.763557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:21.763559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:21.763676 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:21.764122 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:21.764127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:21.764133 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:21.764451 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:21.764635 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:21.765015 (+ 380us) server_negotiation.cc:299] Negotiation successful
0619 06:44:21.765229 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:44:21.765698 17985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:21.750981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:44803 (local address 127.16.79.65:43893)
0619 06:44:21.751598 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:21.751613 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:21.751776 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:21.752113 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:21.752118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:21.752694 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:21.753151 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:21.753171 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:21.754961 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:21.754965 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:21.756671 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:21.756687 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:21.757782 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:21.757787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:21.757909 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:21.758551 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:21.758567 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:21.759271 (+ 704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:21.762817 (+ 3546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:21.762824 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:21.762831 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:21.763425 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:21.763792 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:21.763797 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:21.763800 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:21.763984 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:21.764584 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:21.764589 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:21.764780 (+ 191us) client_negotiation.cc:769] Sending connection context
0619 06:44:21.764924 (+ 144us) client_negotiation.cc:240] Negotiation successful
0619 06:44:21.765239 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":373,"threads_started":1}
I20260619 06:44:21.776405 17895 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:45653
I20260619 06:44:21.776434 18030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:45653 every 8 connection(s)
I20260619 06:44:21.777608 17895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:44:21.779769 18031 heartbeater.cc:347] Connected to a master server at 127.16.79.126:44803
I20260619 06:44:21.779992 18031 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:21.780661 18031 heartbeater.cc:511] Master 127.16.79.126:44803 requested a full tablet report, sending...
I20260619 06:44:21.780740 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 17895
I20260619 06:44:21.780977 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:44:21.781711 16701 external_mini_cluster.cc:1428] Setting key 41518d2699b196108110b90085c68c99
I20260619 06:44:21.783073 17824 ts_manager.cc:194] Registered new tserver with Master: a091f347648c490abd6a8c2a7d0da576 (127.16.79.65:45653)
I20260619 06:44:21.785095 17824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:52987
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:44:21.820910 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:44803
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:38089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:22.132488 18035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:22.132758 18035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:22.132979 18035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:22.140080 18035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:22.140195 18035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:22.140353 18035 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:44:22.140479 18035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:44:22.153159 18035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:38089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.79.126:44803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:22.154637 18035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:22.156301 18035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:22.170516 18043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:22.170490 18040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:22.170583 18041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:22.171115 18035 server_base.cc:1034] running on GCE node
I20260619 06:44:22.171693 18035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:22.172649 18035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:22.173889 18035 hybrid_clock.cc:648] HybridClock initialized: now 1781851462173834 us; error 33 us; skew 500 ppm
I20260619 06:44:22.174185 18035 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:22 dist-test-slave-5w2x krb5kdc[17779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851462, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:22.177338 18035 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:44:22.178737 18035 webserver.cc:460] Webserver started at http://127.16.79.66:38681/ using document root <none> and password file <none>
I20260619 06:44:22.179381 18035 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:22.179463 18035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:22.179721 18035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:22.182655 18035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "7198346571a04355bcb52a1714b8b4ab"
format_stamp: "Formatted at 2026-06-19 06:44:22 on dist-test-slave-5w2x"
server_key: "fd822a5d4f36c73fafd77404278df4bd"
server_key_iv: "e453ea797257d97efdcb3f65b72d6a5f"
server_key_version: "encryptionkey@0"
I20260619 06:44:22.183369 18035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "7198346571a04355bcb52a1714b8b4ab"
format_stamp: "Formatted at 2026-06-19 06:44:22 on dist-test-slave-5w2x"
server_key: "fd822a5d4f36c73fafd77404278df4bd"
server_key_iv: "e453ea797257d97efdcb3f65b72d6a5f"
server_key_version: "encryptionkey@0"
I20260619 06:44:22.188443 18035 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260619 06:44:22.191728 18050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:22.193195 18035 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:44:22.193372 18035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "7198346571a04355bcb52a1714b8b4ab"
format_stamp: "Formatted at 2026-06-19 06:44:22 on dist-test-slave-5w2x"
server_key: "fd822a5d4f36c73fafd77404278df4bd"
server_key_iv: "e453ea797257d97efdcb3f65b72d6a5f"
server_key_version: "encryptionkey@0"
I20260619 06:44:22.193565 18035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:22.214136 18035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:22.218446 18035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:22.218734 18035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:22.221462 18035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:22.221575 18035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:22.221742 18035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:22.221812 18035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 06:44:22 dist-test-slave-5w2x krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851462, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:22.245064 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.221777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:39137 (local address 127.16.79.126:44803)
0619 06:44:22.221957 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:22.221961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:22.223454 (+ 1493us) server_negotiation.cc:407] Connection header received
0619 06:44:22.225486 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:22.225493 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:22.225598 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:22.225716 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:22.228227 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.229296 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.231995 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.232624 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.232782 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.237808 (+ 5026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:22.237816 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:22.237823 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:22.237867 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:22.241170 (+ 3303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:22.242219 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:22.242227 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:22.242232 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:22.242378 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:22.242956 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:22.242962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:22.242967 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:22.243392 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:22.243574 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:22.244311 (+ 737us) server_negotiation.cc:299] Negotiation successful
0619 06:44:22.244631 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 06:44:22.245661 18059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.222493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:44803 (local address 127.16.79.66:39137)
0619 06:44:22.223301 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:22.223336 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:22.225160 (+ 1824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:22.226004 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:22.226020 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:22.226932 (+ 912us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:22.227981 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.228018 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.229520 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.229526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.231827 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.231840 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.232922 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.232931 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.233147 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.233982 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:22.234008 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:22.237535 (+ 3527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:22.241342 (+ 3807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:22.241352 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:22.241375 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:22.242015 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:22.242517 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:22.242520 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:22.242522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:22.242796 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:22.243555 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:22.243564 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:22.243891 (+ 327us) client_negotiation.cc:769] Sending connection context
0619 06:44:22.244109 (+ 218us) client_negotiation.cc:240] Negotiation successful
0619 06:44:22.244533 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":330,"threads_started":1}
I20260619 06:44:22.265834 18118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.251283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:44803 (local address 127.16.79.66:50873)
0619 06:44:22.251869 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:22.251888 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:22.252044 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:22.252390 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:22.252394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:22.253064 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:22.253570 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.253580 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.254566 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.254571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.255734 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.255745 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.256701 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.256711 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.256990 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.257744 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:22.257761 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:22.258816 (+ 1055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:22.262764 (+ 3948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:22.262774 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:22.262782 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:22.263381 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:22.263770 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:22.263774 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:22.263778 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:22.263897 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:22.264540 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:22.264545 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:22.264811 (+ 266us) client_negotiation.cc:769] Sending connection context
0619 06:44:22.265036 (+ 225us) client_negotiation.cc:240] Negotiation successful
0619 06:44:22.265420 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":348,"threads_started":1}
I20260619 06:44:22.266535 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.251181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:50873 (local address 127.16.79.126:44803)
0619 06:44:22.251334 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:22.251338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:22.251977 (+ 639us) server_negotiation.cc:407] Connection header received
0619 06:44:22.252184 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:22.252189 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:22.252264 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:22.252362 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:22.253716 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.254425 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.255865 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.256464 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.256616 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.259032 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:22.259042 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:22.259050 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:22.259098 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:22.262590 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:22.263502 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:22.263510 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:22.263515 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:22.263646 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:22.264013 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:22.264021 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:22.264025 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:22.264413 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:22.264570 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:22.266138 (+ 1568us) server_negotiation.cc:299] Negotiation successful
0619 06:44:22.266338 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 06:44:22.277364 18035 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:46377
I20260619 06:44:22.277385 18169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:46377 every 8 connection(s)
I20260619 06:44:22.278635 18035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:44:22.280812 18170 heartbeater.cc:347] Connected to a master server at 127.16.79.126:44803
I20260619 06:44:22.281013 18170 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:22.281582 18170 heartbeater.cc:511] Master 127.16.79.126:44803 requested a full tablet report, sending...
I20260619 06:44:22.282837 17824 ts_manager.cc:194] Registered new tserver with Master: 7198346571a04355bcb52a1714b8b4ab (127.16.79.66:46377)
I20260619 06:44:22.283653 17824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:39137
I20260619 06:44:22.287380 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 18035
I20260619 06:44:22.287616 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:44:22.288151 16701 external_mini_cluster.cc:1428] Setting key d7a80077651ced1585fd5e2e0da7de97
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:44:22.325625 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:44803
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:38089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:22.637022 18174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:22.637287 18174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:22.637454 18174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:22.644438 18174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:22.644578 18174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:22.644737 18174 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:44:22.644861 18174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:44:22.654500 18174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:38089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.79.126:44803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:22.655736 18174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:22.657130 18174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:22.669715 18179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:22.669669 18180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:22.669740 18182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:22.670661 18174 server_base.cc:1034] running on GCE node
I20260619 06:44:22.671207 18174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:22.672003 18174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:22.673218 18174 hybrid_clock.cc:648] HybridClock initialized: now 1781851462673160 us; error 42 us; skew 500 ppm
I20260619 06:44:22.673514 18174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:22 dist-test-slave-5w2x krb5kdc[17779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851462, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:22.676714 18174 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:44:22.678095 18174 webserver.cc:460] Webserver started at http://127.16.79.67:42069/ using document root <none> and password file <none>
I20260619 06:44:22.678731 18174 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:22.678814 18174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:22.679052 18174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:22.681807 18174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "28f83fed7ac843e5941efbd4879219ea"
format_stamp: "Formatted at 2026-06-19 06:44:22 on dist-test-slave-5w2x"
server_key: "8b63939274d356b7f6e5df46d05cc746"
server_key_iv: "5b930d632fd69e9a7f1189ddf62d5c71"
server_key_version: "encryptionkey@0"
I20260619 06:44:22.682521 18174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "28f83fed7ac843e5941efbd4879219ea"
format_stamp: "Formatted at 2026-06-19 06:44:22 on dist-test-slave-5w2x"
server_key: "8b63939274d356b7f6e5df46d05cc746"
server_key_iv: "5b930d632fd69e9a7f1189ddf62d5c71"
server_key_version: "encryptionkey@0"
I20260619 06:44:22.687403 18174 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260619 06:44:22.690881 18189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:22.692277 18174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 06:44:22.692437 18174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "28f83fed7ac843e5941efbd4879219ea"
format_stamp: "Formatted at 2026-06-19 06:44:22 on dist-test-slave-5w2x"
server_key: "8b63939274d356b7f6e5df46d05cc746"
server_key_iv: "5b930d632fd69e9a7f1189ddf62d5c71"
server_key_version: "encryptionkey@0"
I20260619 06:44:22.692675 18174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:22.709100 18174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:22.713307 18174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:22.713573 18174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:22.716490 18174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:22.716634 18174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:22.716786 18174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:22.716866 18174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 06:44:22 dist-test-slave-5w2x krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851462, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:22.744701 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.717053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:46171 (local address 127.16.79.126:44803)
0619 06:44:22.717286 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:22.717292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:22.718799 (+ 1507us) server_negotiation.cc:407] Connection header received
0619 06:44:22.721996 (+ 3197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:22.722003 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:22.722099 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:22.722233 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:22.724409 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.725768 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.728018 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.728845 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.729005 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.734320 (+ 5315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:22.734330 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:22.734336 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:22.734391 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:22.738087 (+ 3696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:22.739360 (+ 1273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:22.739369 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:22.739375 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:22.739559 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:22.740306 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:22.740314 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:22.740321 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:22.740740 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:22.740918 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:22.743884 (+ 2966us) server_negotiation.cc:299] Negotiation successful
0619 06:44:22.744221 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260619 06:44:22.744731 18198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.717733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:44803 (local address 127.16.79.67:46171)
0619 06:44:22.718594 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:22.718646 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:22.721577 (+ 2931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:22.722522 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:22.722537 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:22.723354 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:22.724190 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.724228 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.725980 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.725991 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.727843 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.727861 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.729077 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.729082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.729262 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.730049 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:22.730086 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:22.733991 (+ 3905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:22.738286 (+ 4295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:22.738299 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:22.738324 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:22.739179 (+ 855us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:22.739737 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:22.739741 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:22.739747 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:22.740073 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:22.741014 (+ 941us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:22.741026 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:22.741663 (+ 637us) client_negotiation.cc:769] Sending connection context
0619 06:44:22.742053 (+ 390us) client_negotiation.cc:240] Negotiation successful
0619 06:44:22.742828 (+ 775us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":336,"threads_started":1}
I20260619 06:44:22.766770 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.752673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:49903 (local address 127.16.79.126:44803)
0619 06:44:22.752915 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:22.752924 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:22.754179 (+ 1255us) server_negotiation.cc:407] Connection header received
0619 06:44:22.754370 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:22.754374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:22.754432 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:22.754510 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:22.755895 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.756789 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.758886 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.759283 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.759420 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.761332 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:22.761336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:22.761339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:22.761367 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:22.763766 (+ 2399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:22.764499 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:22.764507 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:22.764513 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:22.764691 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:22.765219 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:22.765230 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:22.765234 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:22.765470 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:22.765649 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:22.766157 (+ 508us) server_negotiation.cc:299] Negotiation successful
0619 06:44:22.766381 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 06:44:22.766958 18290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.753152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:44803 (local address 127.16.79.67:49903)
0619 06:44:22.754057 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:22.754082 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:22.754249 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:22.754575 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:22.754583 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:22.755283 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:22.755747 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.755763 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.756915 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.756921 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.758729 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.758741 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.759461 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.759467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.759613 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.760391 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:22.760407 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:22.761086 (+ 679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:22.763896 (+ 2810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:22.763902 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:22.763906 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:22.764379 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:22.764864 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:22.764869 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:22.764873 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:22.765047 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:22.765618 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:22.765626 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:22.765861 (+ 235us) client_negotiation.cc:769] Sending connection context
0619 06:44:22.766016 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 06:44:22.766344 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":466,"threads_started":1}
I20260619 06:44:22.769114 18174 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:40015
I20260619 06:44:22.770663 18174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:44:22.769114 18308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:40015 every 8 connection(s)
I20260619 06:44:22.771781 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 18174
I20260619 06:44:22.771974 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:44:22.772449 16701 external_mini_cluster.cc:1428] Setting key a149b9b85ef97c9ddccff56cfa76ed6c
I20260619 06:44:22.772944 18309 heartbeater.cc:347] Connected to a master server at 127.16.79.126:44803
I20260619 06:44:22.773185 18309 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:22.773870 18309 heartbeater.cc:511] Master 127.16.79.126:44803 requested a full tablet report, sending...
I20260619 06:44:22.775333 17824 ts_manager.cc:194] Registered new tserver with Master: 28f83fed7ac843e5941efbd4879219ea (127.16.79.67:40015)
I20260619 06:44:22.776407 17824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:46171
I20260619 06:44:22.787387 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:44:22.788232 18031 heartbeater.cc:503] Master 127.16.79.126:44803 was elected leader, sending a full tablet report...
Jun 19 06:44:22 dist-test-slave-5w2x krb5kdc[17779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851462, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 06:44:22 dist-test-slave-5w2x krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851462, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:22.830811 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.817895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34226 (local address 127.16.79.126:44803)
0619 06:44:22.818124 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:22.818131 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:22.818313 (+ 182us) server_negotiation.cc:407] Connection header received
0619 06:44:22.818459 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:22.818463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:22.818525 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:22.818606 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:22.819933 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.820696 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.821877 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.822294 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.822386 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.825930 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:22.825935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:22.825938 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:22.825968 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:22.828602 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:22.829167 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:22.829170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:22.829172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:22.829282 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:22.829589 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:22.829592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:22.829594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:22.829858 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:22.830032 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:22.830238 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 06:44:22.830395 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 06:44:22.834779 17824 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:44:22.842002 17824 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:44:22.879221 18326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.854454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47262 (local address 127.16.79.67:40015)
0619 06:44:22.871304 (+ 16850us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:22.871312 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:22.871346 (+ 34us) server_negotiation.cc:407] Connection header received
0619 06:44:22.871443 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:22.871453 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:22.871664 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:22.871851 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:22.872942 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.874480 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.877017 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.878063 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.878192 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.878441 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:22.878708 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 06:44:22.878881 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16686,"thread_start_us":17665,"threads_started":1}
I20260619 06:44:22.879290 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.854096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:40015 (local address 127.0.0.1:47262)
0619 06:44:22.864972 (+ 10876us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:22.864995 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:22.865138 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:22.871861 (+ 6723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:22.871869 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:22.871908 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:22.872660 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.872675 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.874616 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.874621 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.876872 (+ 2251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.876883 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.878257 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.878266 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.878454 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.878500 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 06:44:22.878631 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 06:44:22.878850 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10721,"mutex_wait_us":41,"thread_start_us":10581,"threads_started":1}
I20260619 06:44:22.879858 18327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.853710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:46377 (local address 127.0.0.1:53612)
0619 06:44:22.863757 (+ 10047us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:22.863805 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:22.863983 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:22.871833 (+ 7850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:22.871841 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:22.871873 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:22.872549 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.872566 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.874212 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.874220 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.876701 (+ 2481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.876719 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.878707 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.878713 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.878862 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.878891 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 06:44:22.878972 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 06:44:22.879100 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9788,"mutex_wait_us":63,"thread_start_us":18825,"threads_started":1}
I20260619 06:44:22.880911 18325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.854527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37090 (local address 127.16.79.65:45653)
0619 06:44:22.870092 (+ 15565us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:22.870106 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:22.872983 (+ 2877us) server_negotiation.cc:407] Connection header received
0619 06:44:22.873123 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:22.873132 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:22.873361 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:22.873559 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:22.874045 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.875664 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.877880 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.879273 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.879422 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.879614 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:22.880458 (+ 844us) server_negotiation.cc:299] Negotiation successful
0619 06:44:22.880619 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15323,"thread_start_us":15594,"threads_started":1}
I20260619 06:44:22.880914 18322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.854787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:45653 (local address 127.0.0.1:37090)
0619 06:44:22.872849 (+ 18062us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:22.872871 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:22.872987 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:22.873572 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:22.873577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:22.873600 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:22.873894 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.873903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.875830 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.875833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.877699 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.877724 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.879776 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.879783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.879988 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.880021 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 06:44:22.880397 (+ 376us) client_negotiation.cc:240] Negotiation successful
0619 06:44:22.880515 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17943,"mutex_wait_us":2,"thread_start_us":8576,"threads_started":1}
I20260619 06:44:22.880983 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.853893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53612 (local address 127.16.79.66:46377)
0619 06:44:22.870829 (+ 16936us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:22.870841 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:22.870877 (+ 36us) server_negotiation.cc:407] Connection header received
0619 06:44:22.870998 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:22.871007 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:22.871306 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:22.871529 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:22.872804 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.874098 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.876877 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.878194 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.878376 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.878577 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:22.879395 (+ 818us) server_negotiation.cc:299] Negotiation successful
0619 06:44:22.880707 (+ 1312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16716,"thread_start_us":17637,"threads_started":1}
I20260619 06:44:22.883607 18239 tablet_service.cc:1467] Processing CreateTablet for tablet 65c449ff4643420ab598fa614c7b9986 (DEFAULT_TABLE table=test-table [id=daf586f8755f49f78edfd74ff8c9f535]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:22.884277 18100 tablet_service.cc:1467] Processing CreateTablet for tablet 65c449ff4643420ab598fa614c7b9986 (DEFAULT_TABLE table=test-table [id=daf586f8755f49f78edfd74ff8c9f535]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:22.885250 18239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65c449ff4643420ab598fa614c7b9986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:22.885620 18100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65c449ff4643420ab598fa614c7b9986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:22.886919 17961 tablet_service.cc:1467] Processing CreateTablet for tablet 65c449ff4643420ab598fa614c7b9986 (DEFAULT_TABLE table=test-table [id=daf586f8755f49f78edfd74ff8c9f535]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:22.887931 17961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65c449ff4643420ab598fa614c7b9986. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:22.899981 18328 tablet_bootstrap.cc:492] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576: Bootstrap starting.
I20260619 06:44:22.902000 18330 tablet_bootstrap.cc:492] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea: Bootstrap starting.
I20260619 06:44:22.903880 18328 tablet_bootstrap.cc:654] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:22.904816 18329 tablet_bootstrap.cc:492] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab: Bootstrap starting.
I20260619 06:44:22.905211 18328 log.cc:824] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:22.905561 18330 tablet_bootstrap.cc:654] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:22.906595 18330 log.cc:824] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:22.908097 18329 tablet_bootstrap.cc:654] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:22.908268 18328 tablet_bootstrap.cc:492] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576: No bootstrap required, opened a new log
I20260619 06:44:22.908531 18328 ts_tablet_manager.cc:1397] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260619 06:44:22.909281 18329 log.cc:824] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:22.909524 18330 tablet_bootstrap.cc:492] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea: No bootstrap required, opened a new log
I20260619 06:44:22.909744 18330 ts_tablet_manager.cc:1397] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 06:44:22.916973 18329 tablet_bootstrap.cc:492] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab: No bootstrap required, opened a new log
I20260619 06:44:22.917240 18329 ts_tablet_manager.cc:1397] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.005s
I20260619 06:44:22.919579 18328 raft_consensus.cc:348] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } }
I20260619 06:44:22.920020 18328 raft_consensus.cc:374] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:22.920127 18328 raft_consensus.cc:729] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a091f347648c490abd6a8c2a7d0da576, State: Initialized, Role: FOLLOWER
I20260619 06:44:22.920095 18330 raft_consensus.cc:348] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } }
I20260619 06:44:22.920559 18330 raft_consensus.cc:374] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:22.920644 18330 raft_consensus.cc:729] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28f83fed7ac843e5941efbd4879219ea, State: Initialized, Role: FOLLOWER
I20260619 06:44:22.920902 18328 consensus_queue.cc:260] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } }
I20260619 06:44:22.921186 18330 consensus_queue.cc:260] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } }
I20260619 06:44:22.922945 18328 ts_tablet_manager.cc:1428] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576: Time spent starting tablet: real 0.014s user 0.006s sys 0.007s
I20260619 06:44:22.923832 18309 heartbeater.cc:503] Master 127.16.79.126:44803 was elected leader, sending a full tablet report...
I20260619 06:44:22.924515 18330 ts_tablet_manager.cc:1428] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260619 06:44:22.926245 18329 raft_consensus.cc:348] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } }
I20260619 06:44:22.926692 18329 raft_consensus.cc:374] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:22.926796 18329 raft_consensus.cc:729] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7198346571a04355bcb52a1714b8b4ab, State: Initialized, Role: FOLLOWER
I20260619 06:44:22.927397 18329 consensus_queue.cc:260] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } }
I20260619 06:44:22.928591 18170 heartbeater.cc:503] Master 127.16.79.126:44803 was elected leader, sending a full tablet report...
I20260619 06:44:22.929131 18329 ts_tablet_manager.cc:1428] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260619 06:44:22.945597 18335 raft_consensus.cc:482] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:44:22.945932 18335 raft_consensus.cc:504] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } }
I20260619 06:44:22.947930 18335 leader_election.cc:288] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7198346571a04355bcb52a1714b8b4ab (127.16.79.66:46377), a091f347648c490abd6a8c2a7d0da576 (127.16.79.65:45653)
I20260619 06:44:22.954459 18325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.948692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:34599 (local address 127.16.79.65:45653)
0619 06:44:22.948895 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:22.948902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:22.948931 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:44:22.949009 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:22.949014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:22.949102 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:22.949205 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:22.949657 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.950484 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.952454 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.953680 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.953795 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.953847 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:22.954190 (+ 343us) server_negotiation.cc:299] Negotiation successful
0619 06:44:22.954284 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:44:22.954541 18198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.948313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:45653 (local address 127.16.79.67:34599)
0619 06:44:22.948523 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:22.948543 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:22.948689 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:22.949239 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:22.949244 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:22.949279 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:22.949480 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.949492 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.950609 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.950615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.952288 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.952300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.953820 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.953826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.954017 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.954054 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 06:44:22.954148 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 06:44:22.954276 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 06:44:22.955674 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.948276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:34313 (local address 127.16.79.66:46377)
0619 06:44:22.948494 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:22.948500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:22.949329 (+ 829us) server_negotiation.cc:407] Connection header received
0619 06:44:22.949577 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:22.949580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:22.949634 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:22.949730 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:22.950412 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.951471 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.953665 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.954875 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.955041 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.955115 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:22.955420 (+ 305us) server_negotiation.cc:299] Negotiation successful
0619 06:44:22.955506 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:44:22.955693 17986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c449ff4643420ab598fa614c7b9986" candidate_uuid: "28f83fed7ac843e5941efbd4879219ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a091f347648c490abd6a8c2a7d0da576" is_pre_election: true
I20260619 06:44:22.955916 18337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:22.948312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:46377 (local address 127.16.79.67:34313)
0619 06:44:22.949240 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:22.949258 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:22.949428 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:22.949907 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:22.949909 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:22.949927 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:22.950252 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.950260 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.951617 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.951620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.953509 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:22.953524 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:22.955077 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:22.955081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:22.955253 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:22.955291 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 06:44:22.955395 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 06:44:22.955521 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"mutex_wait_us":180,"thread_start_us":391,"threads_started":1}
I20260619 06:44:22.956293 17986 raft_consensus.cc:2454] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28f83fed7ac843e5941efbd4879219ea in term 0.
I20260619 06:44:22.956979 18193 leader_election.cc:302] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28f83fed7ac843e5941efbd4879219ea, a091f347648c490abd6a8c2a7d0da576; no voters:
I20260619 06:44:22.956887 18125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c449ff4643420ab598fa614c7b9986" candidate_uuid: "28f83fed7ac843e5941efbd4879219ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7198346571a04355bcb52a1714b8b4ab" is_pre_election: true
I20260619 06:44:22.957408 18335 raft_consensus.cc:2781] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:44:22.957386 18125 raft_consensus.cc:2454] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28f83fed7ac843e5941efbd4879219ea in term 0.
I20260619 06:44:22.957558 18335 raft_consensus.cc:482] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:44:22.957688 18335 raft_consensus.cc:3037] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:22.959980 18335 raft_consensus.cc:504] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } }
I20260619 06:44:22.960585 18335 leader_election.cc:288] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [CANDIDATE]: Term 1 election: Requested vote from peers 7198346571a04355bcb52a1714b8b4ab (127.16.79.66:46377), a091f347648c490abd6a8c2a7d0da576 (127.16.79.65:45653)
I20260619 06:44:22.961035 18125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c449ff4643420ab598fa614c7b9986" candidate_uuid: "28f83fed7ac843e5941efbd4879219ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7198346571a04355bcb52a1714b8b4ab"
I20260619 06:44:22.961083 17986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c449ff4643420ab598fa614c7b9986" candidate_uuid: "28f83fed7ac843e5941efbd4879219ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a091f347648c490abd6a8c2a7d0da576"
I20260619 06:44:22.961247 18125 raft_consensus.cc:3037] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:22.961282 17986 raft_consensus.cc:3037] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:22.963326 18125 raft_consensus.cc:2454] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28f83fed7ac843e5941efbd4879219ea in term 1.
I20260619 06:44:22.963753 18190 leader_election.cc:302] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28f83fed7ac843e5941efbd4879219ea, 7198346571a04355bcb52a1714b8b4ab; no voters:
I20260619 06:44:22.964092 18335 raft_consensus.cc:2781] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:22.964056 17986 raft_consensus.cc:2454] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28f83fed7ac843e5941efbd4879219ea in term 1.
I20260619 06:44:22.964665 18335 raft_consensus.cc:686] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [term 1 LEADER]: Becoming Leader. State: Replica: 28f83fed7ac843e5941efbd4879219ea, State: Running, Role: LEADER
I20260619 06:44:22.965152 18335 consensus_queue.cc:237] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } }
I20260619 06:44:22.969945 17823 catalog_manager.cc:5495] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 28f83fed7ac843e5941efbd4879219ea (127.16.79.67). New cstate: current_term: 1 leader_uuid: "28f83fed7ac843e5941efbd4879219ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } health_report { overall_health: UNKNOWN } } }
I20260619 06:44:23.010978 18326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.005573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47268 (local address 127.16.79.67:40015)
0619 06:44:23.005770 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:23.005776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:23.005853 (+ 77us) server_negotiation.cc:407] Connection header received
0619 06:44:23.006172 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:23.006177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:23.006260 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:23.006407 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:44:23.006788 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.007481 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.008866 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.009282 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.009420 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.009903 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:44:23.010375 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:44:23.010527 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:23.010644 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 06:44:23.010721 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:44:23.018790 18125 raft_consensus.cc:1261] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab [term 1 FOLLOWER]: Refusing update from remote peer 28f83fed7ac843e5941efbd4879219ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:23.018816 17986 raft_consensus.cc:1261] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576 [term 1 FOLLOWER]: Refusing update from remote peer 28f83fed7ac843e5941efbd4879219ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:23.019779 18335 consensus_queue.cc:1035] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:23.020172 18338 consensus_queue.cc:1035] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 06:44:23.022629 18310 tablet.cc:2369] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:44:23.030047 18032 tablet.cc:2369] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:44:23.030524 18171 tablet.cc:2369] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:44:23.048820 17823 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:44:23.049116 17823 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:44:23.052753 17823 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039: Sending DeleteTablet for 3 replicas of tablet 65c449ff4643420ab598fa614c7b9986
I20260619 06:44:23.053825 18100 tablet_service.cc:1514] Processing DeleteTablet for tablet 65c449ff4643420ab598fa614c7b9986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:23 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:53612
I20260619 06:44:23.053875 18239 tablet_service.cc:1514] Processing DeleteTablet for tablet 65c449ff4643420ab598fa614c7b9986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:23 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:47262
I20260619 06:44:23.054175 17961 tablet_service.cc:1514] Processing DeleteTablet for tablet 65c449ff4643420ab598fa614c7b9986 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:23 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:37090
I20260619 06:44:23.055781 17823 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:44:23.056568 17823 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:44:23.059564 18351 tablet_replica.cc:330] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab: stopping tablet replica
I20260619 06:44:23.060034 18351 raft_consensus.cc:2229] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:44:23.060559 18351 raft_consensus.cc:2258] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:23.063995 18351 ts_tablet_manager.cc:1905] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:23.066157 18238 tablet_service.cc:1467] Processing CreateTablet for tablet 89c8b80041d6434793b6cfa968648d97 (DEFAULT_TABLE table=test-table [id=015bcd01f39c4fd680abbf091413bcd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:23.066383 17960 tablet_service.cc:1467] Processing CreateTablet for tablet 89c8b80041d6434793b6cfa968648d97 (DEFAULT_TABLE table=test-table [id=015bcd01f39c4fd680abbf091413bcd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:23.066870 18238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89c8b80041d6434793b6cfa968648d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:23.066957 17960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89c8b80041d6434793b6cfa968648d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:23.070829 18351 ts_tablet_manager.cc:1918] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:44:23.071002 18351 log.cc:1196] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/65c449ff4643420ab598fa614c7b9986
I20260619 06:44:23.071528 18351 ts_tablet_manager.cc:1939] T 65c449ff4643420ab598fa614c7b9986 P 7198346571a04355bcb52a1714b8b4ab: Deleting consensus metadata
I20260619 06:44:23.071682 18353 tablet_replica.cc:330] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576: stopping tablet replica
I20260619 06:44:23.073598 17808 catalog_manager.cc:4841] TS 7198346571a04355bcb52a1714b8b4ab (127.16.79.66:46377): tablet 65c449ff4643420ab598fa614c7b9986 (table test-table [id=daf586f8755f49f78edfd74ff8c9f535]) successfully deleted
I20260619 06:44:23.073838 18330 tablet_bootstrap.cc:492] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea: Bootstrap starting.
I20260619 06:44:23.074321 18100 tablet_service.cc:1467] Processing CreateTablet for tablet 89c8b80041d6434793b6cfa968648d97 (DEFAULT_TABLE table=test-table [id=015bcd01f39c4fd680abbf091413bcd4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:23.074767 18100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89c8b80041d6434793b6cfa968648d97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:23.075557 18328 tablet_bootstrap.cc:492] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576: Bootstrap starting.
I20260619 06:44:23.075525 18353 raft_consensus.cc:2229] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:44:23.075836 18330 tablet_bootstrap.cc:654] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:23.076123 18353 raft_consensus.cc:2258] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:23.077400 18328 tablet_bootstrap.cc:654] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:23.085740 18353 ts_tablet_manager.cc:1905] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:23.086314 18352 tablet_replica.cc:330] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea: stopping tablet replica
I20260619 06:44:23.086829 18352 raft_consensus.cc:2229] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:44:23.087508 18352 raft_consensus.cc:2258] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:23.087641 18328 tablet_bootstrap.cc:492] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576: No bootstrap required, opened a new log
I20260619 06:44:23.087867 18328 ts_tablet_manager.cc:1397] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260619 06:44:23.089046 18328 raft_consensus.cc:348] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.089354 18328 raft_consensus.cc:374] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:23.089454 18328 raft_consensus.cc:729] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a091f347648c490abd6a8c2a7d0da576, State: Initialized, Role: FOLLOWER
I20260619 06:44:23.089787 18328 consensus_queue.cc:260] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.090559 18330 tablet_bootstrap.cc:492] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea: No bootstrap required, opened a new log
I20260619 06:44:23.090708 18330 ts_tablet_manager.cc:1397] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.011s
I20260619 06:44:23.090891 18329 tablet_bootstrap.cc:492] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab: Bootstrap starting.
I20260619 06:44:23.091064 18328 ts_tablet_manager.cc:1428] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 06:44:23.091468 18330 raft_consensus.cc:348] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.091663 18330 raft_consensus.cc:374] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:23.091784 18330 raft_consensus.cc:729] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28f83fed7ac843e5941efbd4879219ea, State: Initialized, Role: FOLLOWER
I20260619 06:44:23.092059 18330 consensus_queue.cc:260] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.092478 18329 tablet_bootstrap.cc:654] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:23.092635 18330 ts_tablet_manager.cc:1428] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 06:44:23.095886 18352 ts_tablet_manager.cc:1905] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:23.101562 18353 ts_tablet_manager.cc:1918] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:44:23.101745 18353 log.cc:1196] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/65c449ff4643420ab598fa614c7b9986
I20260619 06:44:23.102267 18353 ts_tablet_manager.cc:1939] T 65c449ff4643420ab598fa614c7b9986 P a091f347648c490abd6a8c2a7d0da576: Deleting consensus metadata
I20260619 06:44:23.103982 18329 tablet_bootstrap.cc:492] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab: No bootstrap required, opened a new log
I20260619 06:44:23.104246 18329 ts_tablet_manager.cc:1397] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.006s
I20260619 06:44:23.104727 18352 ts_tablet_manager.cc:1918] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:44:23.104940 18352 log.cc:1196] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/65c449ff4643420ab598fa614c7b9986
I20260619 06:44:23.105576 18352 ts_tablet_manager.cc:1939] T 65c449ff4643420ab598fa614c7b9986 P 28f83fed7ac843e5941efbd4879219ea: Deleting consensus metadata
I20260619 06:44:23.106179 17811 catalog_manager.cc:4841] TS a091f347648c490abd6a8c2a7d0da576 (127.16.79.65:45653): tablet 65c449ff4643420ab598fa614c7b9986 (table test-table [id=daf586f8755f49f78edfd74ff8c9f535]) successfully deleted
I20260619 06:44:23.106905 18329 raft_consensus.cc:348] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.107141 18329 raft_consensus.cc:374] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:23.107239 18329 raft_consensus.cc:729] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7198346571a04355bcb52a1714b8b4ab, State: Initialized, Role: FOLLOWER
I20260619 06:44:23.107460 17809 catalog_manager.cc:4841] TS 28f83fed7ac843e5941efbd4879219ea (127.16.79.67:40015): tablet 65c449ff4643420ab598fa614c7b9986 (table test-table [id=daf586f8755f49f78edfd74ff8c9f535]) successfully deleted
I20260619 06:44:23.107506 18329 consensus_queue.cc:260] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.108064 18329 ts_tablet_manager.cc:1428] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260619 06:44:23.246954 18335 raft_consensus.cc:482] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:44:23.247216 18335 raft_consensus.cc:504] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.247979 18335 leader_election.cc:288] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a091f347648c490abd6a8c2a7d0da576 (127.16.79.65:45653), 7198346571a04355bcb52a1714b8b4ab (127.16.79.66:46377)
I20260619 06:44:23.248600 18125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c8b80041d6434793b6cfa968648d97" candidate_uuid: "28f83fed7ac843e5941efbd4879219ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7198346571a04355bcb52a1714b8b4ab" is_pre_election: true
I20260619 06:44:23.248611 17986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c8b80041d6434793b6cfa968648d97" candidate_uuid: "28f83fed7ac843e5941efbd4879219ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a091f347648c490abd6a8c2a7d0da576" is_pre_election: true
I20260619 06:44:23.248840 18125 raft_consensus.cc:2454] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28f83fed7ac843e5941efbd4879219ea in term 0.
I20260619 06:44:23.248848 17986 raft_consensus.cc:2454] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28f83fed7ac843e5941efbd4879219ea in term 0.
I20260619 06:44:23.249264 18190 leader_election.cc:302] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28f83fed7ac843e5941efbd4879219ea, 7198346571a04355bcb52a1714b8b4ab; no voters:
I20260619 06:44:23.249529 18335 raft_consensus.cc:2781] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:44:23.249701 18335 raft_consensus.cc:482] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:44:23.249818 18335 raft_consensus.cc:3037] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:23.251910 18335 raft_consensus.cc:504] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.252584 18335 leader_election.cc:288] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [CANDIDATE]: Term 1 election: Requested vote from peers a091f347648c490abd6a8c2a7d0da576 (127.16.79.65:45653), 7198346571a04355bcb52a1714b8b4ab (127.16.79.66:46377)
I20260619 06:44:23.253054 17986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c8b80041d6434793b6cfa968648d97" candidate_uuid: "28f83fed7ac843e5941efbd4879219ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a091f347648c490abd6a8c2a7d0da576"
I20260619 06:44:23.253085 18125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c8b80041d6434793b6cfa968648d97" candidate_uuid: "28f83fed7ac843e5941efbd4879219ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7198346571a04355bcb52a1714b8b4ab"
I20260619 06:44:23.253253 17986 raft_consensus.cc:3037] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:23.253279 18125 raft_consensus.cc:3037] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:23.255199 17986 raft_consensus.cc:2454] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28f83fed7ac843e5941efbd4879219ea in term 1.
I20260619 06:44:23.255630 18193 leader_election.cc:302] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28f83fed7ac843e5941efbd4879219ea, a091f347648c490abd6a8c2a7d0da576; no voters:
I20260619 06:44:23.255896 18335 raft_consensus.cc:2781] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:23.256165 18125 raft_consensus.cc:2454] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28f83fed7ac843e5941efbd4879219ea in term 1.
I20260619 06:44:23.256210 18335 raft_consensus.cc:686] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [term 1 LEADER]: Becoming Leader. State: Replica: 28f83fed7ac843e5941efbd4879219ea, State: Running, Role: LEADER
I20260619 06:44:23.256683 18335 consensus_queue.cc:237] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.261561 17823 catalog_manager.cc:5495] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 28f83fed7ac843e5941efbd4879219ea (127.16.79.67). New cstate: current_term: 1 leader_uuid: "28f83fed7ac843e5941efbd4879219ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } health_report { overall_health: HEALTHY } } }
Jun 19 06:44:23 dist-test-slave-5w2x krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851461, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:23.317847 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.302730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34240 (local address 127.16.79.126:44803)
0619 06:44:23.302954 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:23.302961 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:23.303572 (+ 611us) server_negotiation.cc:407] Connection header received
0619 06:44:23.303739 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:23.303745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:23.303874 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:23.303992 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:23.305274 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.306395 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.307664 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.308396 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.308531 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.311929 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:23.311934 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:23.311937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:23.311964 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:23.314684 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:23.315383 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:23.315390 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:23.315397 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:23.315527 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:23.316304 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:23.316311 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:23.316314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:23.316619 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:23.316769 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:23.317193 (+ 424us) server_negotiation.cc:299] Negotiation successful
0619 06:44:23.317528 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":145,"server-negotiator.queue_time_us":57}
I20260619 06:44:23.317878 18363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.302768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:44803 (local address 127.0.0.1:34240)
0619 06:44:23.303459 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:23.303476 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:23.303600 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:23.304112 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:23.304116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:23.304830 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:23.305104 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:23.305113 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.306538 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.306542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:23.307513 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:23.307523 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.308597 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.308601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:23.308746 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.309483 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:23.309508 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:23.311739 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:23.314809 (+ 3070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:23.314843 (+ 34us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:23.314852 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:23.315265 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:23.315640 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:23.315645 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:23.315648 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:23.316175 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:23.316767 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:23.316776 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:23.316952 (+ 176us) client_negotiation.cc:769] Sending connection context
0619 06:44:23.317111 (+ 159us) client_negotiation.cc:240] Negotiation successful
0619 06:44:23.317410 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":378,"threads_started":1}
I20260619 06:44:23.322904 17823 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:34240:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 06:44:23.323941 17823 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:44:23.330750 17960 tablet_service.cc:1467] Processing CreateTablet for tablet 707d9919daec46c59fd2f977e39ce316 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3177e305fc4a4a88b97b7ccbe4b1a362]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:44:23.331290 17960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707d9919daec46c59fd2f977e39ce316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:23.331178 18100 tablet_service.cc:1467] Processing CreateTablet for tablet 707d9919daec46c59fd2f977e39ce316 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3177e305fc4a4a88b97b7ccbe4b1a362]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:44:23.331643 18100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707d9919daec46c59fd2f977e39ce316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:23.332674 18239 tablet_service.cc:1467] Processing CreateTablet for tablet 707d9919daec46c59fd2f977e39ce316 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3177e305fc4a4a88b97b7ccbe4b1a362]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:44:23.333200 18239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707d9919daec46c59fd2f977e39ce316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:23.338745 18329 tablet_bootstrap.cc:492] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab: Bootstrap starting.
I20260619 06:44:23.340453 18330 tablet_bootstrap.cc:492] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea: Bootstrap starting.
I20260619 06:44:23.340453 18329 tablet_bootstrap.cc:654] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:23.342485 18330 tablet_bootstrap.cc:654] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:23.343703 18328 tablet_bootstrap.cc:492] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576: Bootstrap starting.
I20260619 06:44:23.345247 18328 tablet_bootstrap.cc:654] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:23.348788 18328 tablet_bootstrap.cc:492] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576: No bootstrap required, opened a new log
I20260619 06:44:23.349059 18328 ts_tablet_manager.cc:1397] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260619 06:44:23.351200 18328 raft_consensus.cc:348] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.351606 18328 raft_consensus.cc:374] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:23.351758 18328 raft_consensus.cc:729] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a091f347648c490abd6a8c2a7d0da576, State: Initialized, Role: FOLLOWER
I20260619 06:44:23.352114 18328 consensus_queue.cc:260] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.352926 18328 ts_tablet_manager.cc:1428] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260619 06:44:23.352739 18334 tablet_replica.cc:439] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } }
I20260619 06:44:23.353108 18334 tablet_replica.cc:442] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:44:23.355999 18329 tablet_bootstrap.cc:492] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab: No bootstrap required, opened a new log
I20260619 06:44:23.356272 18329 ts_tablet_manager.cc:1397] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.006s
I20260619 06:44:23.356773 18330 tablet_bootstrap.cc:492] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea: No bootstrap required, opened a new log
I20260619 06:44:23.356943 18330 ts_tablet_manager.cc:1397] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.002s
I20260619 06:44:23.357097 18329 raft_consensus.cc:348] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.357332 18329 raft_consensus.cc:374] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:23.357409 18329 raft_consensus.cc:729] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7198346571a04355bcb52a1714b8b4ab, State: Initialized, Role: FOLLOWER
I20260619 06:44:23.357648 18329 consensus_queue.cc:260] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.358256 18329 ts_tablet_manager.cc:1428] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260619 06:44:23.358122 18336 tablet_replica.cc:439] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } }
I20260619 06:44:23.358494 18336 tablet_replica.cc:442] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:44:23.357762 18330 raft_consensus.cc:348] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.358749 18330 raft_consensus.cc:374] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:23.358857 18330 raft_consensus.cc:729] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28f83fed7ac843e5941efbd4879219ea, State: Initialized, Role: FOLLOWER
I20260619 06:44:23.359141 18330 consensus_queue.cc:260] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.359557 18335 tablet_replica.cc:439] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } }
I20260619 06:44:23.359879 18335 tablet_replica.cc:442] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:44:23.360919 18330 ts_tablet_manager.cc:1428] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260619 06:44:23.393885 18334 raft_consensus.cc:482] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:44:23.394164 18334 raft_consensus.cc:504] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.395749 18334 leader_election.cc:288] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7198346571a04355bcb52a1714b8b4ab (127.16.79.66:46377), 28f83fed7ac843e5941efbd4879219ea (127.16.79.67:40015)
I20260619 06:44:23.405112 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.396754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:43321 (local address 127.16.79.66:46377)
0619 06:44:23.396960 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:23.396968 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:23.398200 (+ 1232us) server_negotiation.cc:407] Connection header received
0619 06:44:23.398383 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:23.398387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:23.398441 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:23.398534 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:23.399294 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.400462 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.402920 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.403838 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.403992 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.404074 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:23.404674 (+ 600us) server_negotiation.cc:299] Negotiation successful
0619 06:44:23.404813 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:44:23.405507 18326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.396252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:33275 (local address 127.16.79.67:40015)
0619 06:44:23.396471 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:23.396481 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:23.397775 (+ 1294us) server_negotiation.cc:407] Connection header received
0619 06:44:23.398114 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:23.398120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:23.398225 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:23.398338 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:23.399378 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.400123 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.402843 (+ 2720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.404283 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.404432 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.404498 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:23.405076 (+ 578us) server_negotiation.cc:299] Negotiation successful
0619 06:44:23.405240 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:44:23.405627 18367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.395996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:40015 (local address 127.16.79.65:33275)
0619 06:44:23.397656 (+ 1660us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:23.397681 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:23.397962 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:23.398709 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:23.398717 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:23.398757 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:23.399167 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:23.399179 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.400299 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.400307 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:23.402671 (+ 2364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:23.402686 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.404681 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.404690 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:23.404864 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.404899 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 06:44:23.405041 (+ 142us) client_negotiation.cc:240] Negotiation successful
0619 06:44:23.405185 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1511,"mutex_wait_us":3,"spinlock_wait_cycles":1408,"thread_start_us":1342,"threads_started":1}
I20260619 06:44:23.406468 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.396092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:46377 (local address 127.16.79.65:43321)
0619 06:44:23.398104 (+ 2012us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:23.398123 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:23.398247 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:23.398550 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:23.398554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:23.398578 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:23.399073 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:23.399085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.400607 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.400612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:23.402671 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:23.402686 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.404265 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.404271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:23.404444 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.404478 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 06:44:23.406083 (+ 1605us) client_negotiation.cc:240] Negotiation successful
0619 06:44:23.406203 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1878,"mutex_wait_us":54,"spinlock_wait_cycles":24192,"thread_start_us":1635,"threads_started":1}
I20260619 06:44:23.406524 18259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707d9919daec46c59fd2f977e39ce316" candidate_uuid: "a091f347648c490abd6a8c2a7d0da576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28f83fed7ac843e5941efbd4879219ea" is_pre_election: true
I20260619 06:44:23.407007 18259 raft_consensus.cc:2454] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a091f347648c490abd6a8c2a7d0da576 in term 0.
I20260619 06:44:23.407204 18125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707d9919daec46c59fd2f977e39ce316" candidate_uuid: "a091f347648c490abd6a8c2a7d0da576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7198346571a04355bcb52a1714b8b4ab" is_pre_election: true
I20260619 06:44:23.407476 18125 raft_consensus.cc:2454] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a091f347648c490abd6a8c2a7d0da576 in term 0.
I20260619 06:44:23.407694 17912 leader_election.cc:302] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28f83fed7ac843e5941efbd4879219ea, a091f347648c490abd6a8c2a7d0da576; no voters:
I20260619 06:44:23.408114 18334 raft_consensus.cc:2781] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:44:23.408295 18334 raft_consensus.cc:482] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:44:23.408393 18334 raft_consensus.cc:3037] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:23.410398 18334 raft_consensus.cc:504] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.410984 18334 leader_election.cc:288] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [CANDIDATE]: Term 1 election: Requested vote from peers 7198346571a04355bcb52a1714b8b4ab (127.16.79.66:46377), 28f83fed7ac843e5941efbd4879219ea (127.16.79.67:40015)
I20260619 06:44:23.411415 18125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707d9919daec46c59fd2f977e39ce316" candidate_uuid: "a091f347648c490abd6a8c2a7d0da576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7198346571a04355bcb52a1714b8b4ab"
I20260619 06:44:23.411656 18125 raft_consensus.cc:3037] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:23.411657 18259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707d9919daec46c59fd2f977e39ce316" candidate_uuid: "a091f347648c490abd6a8c2a7d0da576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28f83fed7ac843e5941efbd4879219ea"
I20260619 06:44:23.411870 18259 raft_consensus.cc:3037] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:23.414619 18125 raft_consensus.cc:2454] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a091f347648c490abd6a8c2a7d0da576 in term 1.
I20260619 06:44:23.415071 17911 leader_election.cc:302] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7198346571a04355bcb52a1714b8b4ab, a091f347648c490abd6a8c2a7d0da576; no voters:
I20260619 06:44:23.415134 18259 raft_consensus.cc:2454] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a091f347648c490abd6a8c2a7d0da576 in term 1.
I20260619 06:44:23.415453 18334 raft_consensus.cc:2781] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:23.415917 18334 raft_consensus.cc:686] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [term 1 LEADER]: Becoming Leader. State: Replica: a091f347648c490abd6a8c2a7d0da576, State: Running, Role: LEADER
I20260619 06:44:23.416342 18334 consensus_queue.cc:237] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } }
I20260619 06:44:23.418265 18369 tablet_replica.cc:439] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576: TxnStatusTablet state changed. Reason: New leader a091f347648c490abd6a8c2a7d0da576. Latest consensus state: current_term: 1 leader_uuid: "a091f347648c490abd6a8c2a7d0da576" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } }
I20260619 06:44:23.418725 18369 tablet_replica.cc:442] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:44:23.419445 18371 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 06:44:23.419876 18371 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 06:44:23.420235 17824 catalog_manager.cc:5495] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 reported cstate change: term changed from 0 to 1, leader changed from <none> to a091f347648c490abd6a8c2a7d0da576 (127.16.79.65). New cstate: current_term: 1 leader_uuid: "a091f347648c490abd6a8c2a7d0da576" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } health_report { overall_health: UNKNOWN } } }
I20260619 06:44:23.456277 18363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.448892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:45653 (local address 127.0.0.1:37106)
0619 06:44:23.449209 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:23.449229 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:23.449375 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:23.449900 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:23.449906 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:23.449919 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:44:23.450115 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:23.450126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.451614 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.451620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:23.453255 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:23.453269 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.454261 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.454268 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:23.454464 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.454541 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:44:23.455647 (+ 1106us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:44:23.455685 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 06:44:23.455785 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 06:44:23.455942 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164}
I20260619 06:44:23.456300 18325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.448973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37106 (local address 127.16.79.65:45653)
0619 06:44:23.449163 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:23.449169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:23.449309 (+ 140us) server_negotiation.cc:407] Connection header received
0619 06:44:23.449588 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:23.449594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:23.449709 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:23.449865 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:44:23.450260 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.451469 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.453408 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.454080 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.454233 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.454768 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:44:23.455511 (+ 743us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:44:23.455702 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:23.455826 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 06:44:23.455919 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:44:23.461139 18125 raft_consensus.cc:1261] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab [term 1 FOLLOWER]: Refusing update from remote peer a091f347648c490abd6a8c2a7d0da576: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:23.461369 18259 raft_consensus.cc:1261] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea [term 1 FOLLOWER]: Refusing update from remote peer a091f347648c490abd6a8c2a7d0da576: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:23.462050 18369 consensus_queue.cc:1035] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:23.462376 18334 consensus_queue.cc:1035] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:23.476850 18336 tablet_replica.cc:439] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab: TxnStatusTablet state changed. Reason: New leader a091f347648c490abd6a8c2a7d0da576. Latest consensus state: current_term: 1 leader_uuid: "a091f347648c490abd6a8c2a7d0da576" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } }
I20260619 06:44:23.477187 18336 tablet_replica.cc:442] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:44:23.477602 18335 tablet_replica.cc:439] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea: TxnStatusTablet state changed. Reason: New leader a091f347648c490abd6a8c2a7d0da576. Latest consensus state: current_term: 1 leader_uuid: "a091f347648c490abd6a8c2a7d0da576" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } }
I20260619 06:44:23.477898 18335 tablet_replica.cc:442] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:44:23.480243 18369 tablet_replica.cc:439] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a091f347648c490abd6a8c2a7d0da576" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } }
I20260619 06:44:23.480499 18369 tablet_replica.cc:442] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:44:23.480682 18342 mvcc.cc:205] Tried to move back new op lower bound from 7298463594328870912 to 7298463594157019136. Current Snapshot: MvccSnapshot[applied={T|T < 7298463594328870912 or (T in {7298463594328870912})}]
I20260619 06:44:23.482048 18341 mvcc.cc:205] Tried to move back new op lower bound from 7298463594328870912 to 7298463594157019136. Current Snapshot: MvccSnapshot[applied={T|T < 7298463594328870912}]
I20260619 06:44:23.482702 18335 tablet_replica.cc:439] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a091f347648c490abd6a8c2a7d0da576" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } }
I20260619 06:44:23.482977 18335 tablet_replica.cc:442] T 707d9919daec46c59fd2f977e39ce316 P 28f83fed7ac843e5941efbd4879219ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:44:23.489753 18334 tablet_replica.cc:439] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a091f347648c490abd6a8c2a7d0da576" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } }
I20260619 06:44:23.490068 18334 tablet_replica.cc:442] T 707d9919daec46c59fd2f977e39ce316 P a091f347648c490abd6a8c2a7d0da576: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:44:23.491645 18336 tablet_replica.cc:439] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a091f347648c490abd6a8c2a7d0da576" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 } } peers { permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 } } peers { permanent_uuid: "28f83fed7ac843e5941efbd4879219ea" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40015 } } }
I20260619 06:44:23.491870 18336 tablet_replica.cc:442] T 707d9919daec46c59fd2f977e39ce316 P 7198346571a04355bcb52a1714b8b4ab: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:44:23.509995 18325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.502843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:59105 (local address 127.16.79.65:45653)
0619 06:44:23.503003 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:23.503009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:23.503395 (+ 386us) server_negotiation.cc:407] Connection header received
0619 06:44:23.503610 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:23.503616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:23.503702 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:23.503804 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:44:23.504429 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.505633 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.507797 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.508494 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.508612 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.509175 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:44:23.509306 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:44:23.509404 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:23.509566 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 06:44:23.509656 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 06:44:23.511508 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.502440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:45653 (local address 127.16.79.67:59105)
0619 06:44:23.503279 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:23.503300 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:23.503442 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:23.503887 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:23.503892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:23.503905 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:44:23.504252 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:23.504263 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.505804 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.505810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:23.507633 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:23.507649 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.508695 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.508702 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:23.508917 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.508989 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:44:23.509423 (+ 434us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:44:23.509435 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 06:44:23.509519 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 06:44:23.509667 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":572,"threads_started":1}
I20260619 06:44:23.522384 18125 raft_consensus.cc:1261] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab [term 1 FOLLOWER]: Refusing update from remote peer 28f83fed7ac843e5941efbd4879219ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:23.522631 17986 raft_consensus.cc:1261] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576 [term 1 FOLLOWER]: Refusing update from remote peer 28f83fed7ac843e5941efbd4879219ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:23.522994 18346 consensus_queue.cc:1035] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "7198346571a04355bcb52a1714b8b4ab" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:23.524499 18346 consensus_queue.cc:1035] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "a091f347648c490abd6a8c2a7d0da576" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 45653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:23.539678 18343 mvcc.cc:205] Tried to move back new op lower bound from 7298463594580779008 to 7298463593502720000. Current Snapshot: MvccSnapshot[applied={T|T < 7298463594580779008}]
I20260619 06:44:23.570958 18326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.564343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:38595 (local address 127.16.79.67:40015)
0619 06:44:23.564514 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:23.564520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:23.565145 (+ 625us) server_negotiation.cc:407] Connection header received
0619 06:44:23.565413 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:23.565418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:23.565494 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:23.565552 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:44:23.566291 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.567171 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.569321 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.569766 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.569867 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.570399 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:44:23.570501 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:44:23.570600 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:23.570713 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 06:44:23.570772 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:44:23.571563 18382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.564198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:40015 (local address 127.16.79.65:38595)
0619 06:44:23.565049 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:23.565069 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:23.565257 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:23.565753 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:23.565759 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:23.565777 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:44:23.566147 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:23.566158 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.567299 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.567304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:23.569190 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:23.569207 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.569950 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.569956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:23.570152 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.570225 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:44:23.570605 (+ 380us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:44:23.570616 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 06:44:23.570710 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 06:44:23.570856 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":493,"threads_started":1}
I20260619 06:44:23.613849 17823 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:44:23.614100 17823 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:44:23.617053 17823 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 266db23748be4361a504cebbd39fb039: Sending DeleteTablet for 3 replicas of tablet 89c8b80041d6434793b6cfa968648d97
I20260619 06:44:23.617856 17960 tablet_service.cc:1514] Processing DeleteTablet for tablet 89c8b80041d6434793b6cfa968648d97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:23 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:37090
I20260619 06:44:23.617878 18100 tablet_service.cc:1514] Processing DeleteTablet for tablet 89c8b80041d6434793b6cfa968648d97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:23 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:53612
I20260619 06:44:23.618285 18239 tablet_service.cc:1514] Processing DeleteTablet for tablet 89c8b80041d6434793b6cfa968648d97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:23 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:47262
I20260619 06:44:23.630270 18385 tablet_replica.cc:330] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab: stopping tablet replica
I20260619 06:44:23.630708 18385 raft_consensus.cc:2229] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:44:23.631094 18385 raft_consensus.cc:2258] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:23.632527 18385 ts_tablet_manager.cc:1905] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:23.639082 18387 tablet_replica.cc:330] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea: stopping tablet replica
I20260619 06:44:23.639325 18387 raft_consensus.cc:2229] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:44:23.639716 18387 raft_consensus.cc:2258] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:23.640868 18387 ts_tablet_manager.cc:1905] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:23.641438 18385 ts_tablet_manager.cc:1918] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:44:23.641626 18385 log.cc:1196] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/89c8b80041d6434793b6cfa968648d97
I20260619 06:44:23.642150 18385 ts_tablet_manager.cc:1939] T 89c8b80041d6434793b6cfa968648d97 P 7198346571a04355bcb52a1714b8b4ab: Deleting consensus metadata
I20260619 06:44:23.643271 17808 catalog_manager.cc:4841] TS 7198346571a04355bcb52a1714b8b4ab (127.16.79.66:46377): tablet 89c8b80041d6434793b6cfa968648d97 (table test-table [id=015bcd01f39c4fd680abbf091413bcd4]) successfully deleted
I20260619 06:44:23.645849 18386 tablet_replica.cc:330] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576: stopping tablet replica
I20260619 06:44:23.646121 18386 raft_consensus.cc:2229] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:44:23.646552 18386 raft_consensus.cc:2258] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:23.647111 18387 ts_tablet_manager.cc:1918] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:44:23.647293 18387 log.cc:1196] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/89c8b80041d6434793b6cfa968648d97
I20260619 06:44:23.647709 18387 ts_tablet_manager.cc:1939] T 89c8b80041d6434793b6cfa968648d97 P 28f83fed7ac843e5941efbd4879219ea: Deleting consensus metadata
I20260619 06:44:23.647970 18386 ts_tablet_manager.cc:1905] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:23.648876 17809 catalog_manager.cc:4841] TS 28f83fed7ac843e5941efbd4879219ea (127.16.79.67:40015): tablet 89c8b80041d6434793b6cfa968648d97 (table test-table [id=015bcd01f39c4fd680abbf091413bcd4]) successfully deleted
I20260619 06:44:23.653970 18386 ts_tablet_manager.cc:1918] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:44:23.654119 18386 log.cc:1196] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/89c8b80041d6434793b6cfa968648d97
I20260619 06:44:23.654547 18386 ts_tablet_manager.cc:1939] T 89c8b80041d6434793b6cfa968648d97 P a091f347648c490abd6a8c2a7d0da576: Deleting consensus metadata
I20260619 06:44:23.655627 17811 catalog_manager.cc:4841] TS a091f347648c490abd6a8c2a7d0da576 (127.16.79.65:45653): tablet 89c8b80041d6434793b6cfa968648d97 (table test-table [id=015bcd01f39c4fd680abbf091413bcd4]) successfully deleted
Jun 19 06:44:23 dist-test-slave-5w2x krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851462, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:44:23.666313 18325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.640260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37118 (local address 127.16.79.65:45653)
0619 06:44:23.640471 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:23.640477 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:23.649675 (+ 9198us) server_negotiation.cc:407] Connection header received
0619 06:44:23.649869 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:23.649878 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:23.649969 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:23.650090 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:23.651891 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.653169 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.654941 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.655610 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.655747 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.660535 (+ 4788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:23.660558 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:23.660572 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:23.660605 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:23.663533 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:23.664262 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:23.664271 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:23.664275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:23.664402 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:23.664796 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:23.664801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:23.664804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:23.665170 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:23.665534 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:23.665779 (+ 245us) server_negotiation.cc:299] Negotiation successful
0619 06:44:23.665999 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260619 06:44:23.667055 18026 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37118
I20260619 06:44:23.681797 17919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.669847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34244 (local address 127.16.79.126:44803)
0619 06:44:23.670062 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:23.670069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:23.670303 (+ 234us) server_negotiation.cc:407] Connection header received
0619 06:44:23.670485 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:23.670489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:23.670548 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:23.670647 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:23.671950 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.672761 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.673916 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.674312 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.674442 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.676721 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:23.676726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:23.676729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:23.676762 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:23.679189 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:23.679844 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:23.679849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:23.679852 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:23.680002 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:23.680411 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:23.680416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:23.680419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:23.680766 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:23.681013 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:23.681242 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 06:44:23.681461 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260619 06:44:23.682377 17823 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34244
I20260619 06:44:23.684231 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 17895
I20260619 06:44:23.697978 18325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.685054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37120 (local address 127.16.79.65:45653)
0619 06:44:23.685258 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:23.685269 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:23.686026 (+ 757us) server_negotiation.cc:407] Connection header received
0619 06:44:23.686219 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:23.686224 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:23.686310 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:23.686418 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:23.687618 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.688530 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.689724 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.690131 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.690277 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.692395 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:23.692401 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:23.692406 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:23.692446 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:23.695188 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:23.695842 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:23.695849 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:23.695853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:23.695983 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:23.696391 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:23.696398 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:23.696401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:23.696816 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:23.697044 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:23.697415 (+ 371us) server_negotiation.cc:299] Negotiation successful
0619 06:44:23.697640 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260619 06:44:23.698674 18026 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37120
W20260619 06:44:23.699198 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 17895: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:44:23.699366 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 17895
I20260619 06:44:23.717693 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 18035
Jun 19 06:44:23 dist-test-slave-5w2x krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851462, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:44:23.731891 18323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.718430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53624 (local address 127.16.79.66:46377)
0619 06:44:23.718612 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:23.718621 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:23.718642 (+ 21us) server_negotiation.cc:407] Connection header received
0619 06:44:23.718857 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:23.718862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:23.718938 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:23.719059 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:23.720124 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.720866 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.722074 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.722480 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.722618 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.726613 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:23.726632 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:23.726647 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:23.726677 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:23.729267 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:23.729902 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:23.729917 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:23.729920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:23.730036 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:23.730404 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:23.730411 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:23.730414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:23.730737 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:23.731043 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:23.731308 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 06:44:23.731520 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260619 06:44:23.732650 18165 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53624
W20260619 06:44:23.733270 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 18035: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:44:23.733455 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 18035
I20260619 06:44:23.750156 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 18174
Jun 19 06:44:23 dist-test-slave-5w2x krb5kdc[17779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851462, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:44:23.765123 18326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:23.750921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47280 (local address 127.16.79.67:40015)
0619 06:44:23.751080 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:23.751085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:23.751102 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:44:23.751235 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:23.751239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:23.751306 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:23.751389 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:23.752623 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.753318 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.754556 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:23.754955 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:23.755043 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:23.759340 (+ 4297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:23.759356 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:23.759373 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:23.759409 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:23.762295 (+ 2886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:23.762998 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:23.763011 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:23.763013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:23.763142 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:23.763527 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:23.763533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:23.763537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:23.763901 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:23.764277 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:23.764561 (+ 284us) server_negotiation.cc:299] Negotiation successful
0619 06:44:23.764797 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260619 06:44:23.765782 18304 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47280
W20260619 06:44:23.766343 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 18174: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:44:23.766470 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 18174
I20260619 06:44:23.787273 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 17792
I20260619 06:44:23.788410 17864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:23.901012 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 17792
2026-06-19T06:44:23Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4432 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:44:23 dist-test-slave-5w2x krb5kdc[18396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:44:23 dist-test-slave-5w2x krb5kdc[18396](info): set up 2 sockets
Jun 19 06:44:23 dist-test-slave-5w2x krb5kdc[18396](info): commencing operation
krb5kdc: starting...
W20260619 06:44:24.980710 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:44:25 dist-test-slave-5w2x krb5kdc[18396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:44:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:44:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:44:25.159667 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:43827
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:43827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:25.476513 18409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:25.476823 18409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:25.476998 18409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:25.483990 18409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:44:25.484087 18409 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 06:44:25.484191 18409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:25.484308 18409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:25.484396 18409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:44:25.484488 18409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:44:25.493930 18409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:43827
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:43827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:25.495162 18409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:25.496490 18409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:25.508014 18417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:25.508045 18414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:25.507977 18415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:25.508724 18409 server_base.cc:1034] running on GCE node
I20260619 06:44:25.509513 18409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:25.510818 18409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:25.512025 18409 hybrid_clock.cc:648] HybridClock initialized: now 1781851465511970 us; error 29 us; skew 500 ppm
I20260619 06:44:25.512374 18409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:25 dist-test-slave-5w2x krb5kdc[18396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851465, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:25.515347 18409 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:44:25.516705 18409 webserver.cc:460] Webserver started at http://127.16.79.126:46639/ using document root <none> and password file <none>
I20260619 06:44:25.517336 18409 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:25.517422 18409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:25.517678 18409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:25.521142 18409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "93df491966b64691addd7608676ab37a"
format_stamp: "Formatted at 2026-06-19 06:44:25 on dist-test-slave-5w2x"
server_key: "fe90313cfc6df3cf5da857a532b74970"
server_key_iv: "a948b6563cf921bd4647b412f244a24b"
server_key_version: "encryptionkey@0"
I20260619 06:44:25.521867 18409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "93df491966b64691addd7608676ab37a"
format_stamp: "Formatted at 2026-06-19 06:44:25 on dist-test-slave-5w2x"
server_key: "fe90313cfc6df3cf5da857a532b74970"
server_key_iv: "a948b6563cf921bd4647b412f244a24b"
server_key_version: "encryptionkey@0"
I20260619 06:44:25.527069 18409 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260619 06:44:25.530697 18424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:25.532058 18409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:44:25.532342 18409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "93df491966b64691addd7608676ab37a"
format_stamp: "Formatted at 2026-06-19 06:44:25 on dist-test-slave-5w2x"
server_key: "fe90313cfc6df3cf5da857a532b74970"
server_key_iv: "a948b6563cf921bd4647b412f244a24b"
server_key_version: "encryptionkey@0"
I20260619 06:44:25.532572 18409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:25.555835 18409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:25.560714 18409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:25.561004 18409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:25.581681 18409 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:43827
I20260619 06:44:25.581699 18485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:43827 every 8 connection(s)
I20260619 06:44:25.583072 18409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:44:25.586320 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 18409
I20260619 06:44:25.586527 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:44:25.587218 18486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:25.587325 16701 external_mini_cluster.cc:1428] Setting key d4ba1b16d647d9e577827d8f189d635a
I20260619 06:44:25.602167 18486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a: Bootstrap starting.
Jun 19 06:44:25 dist-test-slave-5w2x krb5kdc[18396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:25.606521 18486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:25.607890 18486 log.cc:824] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:25.611542 18486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a: No bootstrap required, opened a new log
I20260619 06:44:25.612355 18489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:25.589469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55716 (local address 127.16.79.126:43827)
0619 06:44:25.590269 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:25.590282 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:25.590330 (+ 48us) server_negotiation.cc:407] Connection header received
0619 06:44:25.592042 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:25.592064 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:25.592534 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:25.592874 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:25.594601 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:25.595871 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:25.597629 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:25.598340 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:25.598488 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:25.603359 (+ 4871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:25.603371 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:25.603391 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:25.603450 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:25.607456 (+ 4006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:25.608318 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:25.608326 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:25.608334 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:25.608581 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:25.609110 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:25.609113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:25.609115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:25.609805 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:25.610088 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:25.610508 (+ 420us) server_negotiation.cc:299] Negotiation successful
0619 06:44:25.610936 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":403,"threads_started":1}
I20260619 06:44:25.619724 18486 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93df491966b64691addd7608676ab37a" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43827 } }
I20260619 06:44:25.620070 18486 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:25.620190 18486 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93df491966b64691addd7608676ab37a, State: Initialized, Role: FOLLOWER
I20260619 06:44:25.620792 18486 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93df491966b64691addd7608676ab37a" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43827 } }
I20260619 06:44:25.621008 18486 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:44:25.621119 18486 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:44:25.621238 18486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:25.623281 18486 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93df491966b64691addd7608676ab37a" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43827 } }
I20260619 06:44:25.623760 18486 leader_election.cc:302] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93df491966b64691addd7608676ab37a; no voters:
I20260619 06:44:25.624372 18486 leader_election.cc:288] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:44:25.624517 18491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:25.625281 18491 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [term 1 LEADER]: Becoming Leader. State: Replica: 93df491966b64691addd7608676ab37a, State: Running, Role: LEADER
I20260619 06:44:25.625840 18491 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93df491966b64691addd7608676ab37a" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43827 } }
I20260619 06:44:25.626600 18486 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:44:25.628729 18493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93df491966b64691addd7608676ab37a. Latest consensus state: current_term: 1 leader_uuid: "93df491966b64691addd7608676ab37a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93df491966b64691addd7608676ab37a" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43827 } } }
I20260619 06:44:25.629048 18493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:25.629911 18498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:44:25.629765 18492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93df491966b64691addd7608676ab37a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93df491966b64691addd7608676ab37a" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43827 } } }
I20260619 06:44:25.630064 18492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:25.635556 18498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:44:25.644759 18498 catalog_manager.cc:1347] Generated new cluster ID: df5ba527a6094a60bf145142306fa5b0
I20260619 06:44:25.644886 18498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:44:25.669332 18498 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:44:25.670619 18498 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:44:25.680902 18498 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a: Generated new TSK 0
I20260619 06:44:25.681749 18498 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:44:25.737955 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43827
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:26.061882 18512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:26.062147 18512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:26.062249 18512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:26.069321 18512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:26.069425 18512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:26.069509 18512 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 06:44:26.069561 18512 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:44:26.069628 18512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:44:26.078985 18512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.79.126:43827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:26.080190 18512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:26.081444 18512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:26.093683 18517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:26.093617 18518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:26.094084 18520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:26.094550 18512 server_base.cc:1034] running on GCE node
I20260619 06:44:26.095100 18512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:26.096103 18512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:26.097371 18512 hybrid_clock.cc:648] HybridClock initialized: now 1781851466097317 us; error 52 us; skew 500 ppm
I20260619 06:44:26.097666 18512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:26 dist-test-slave-5w2x krb5kdc[18396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851466, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:26.105707 18512 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:44:26.107118 18512 webserver.cc:460] Webserver started at http://127.16.79.65:45375/ using document root <none> and password file <none>
I20260619 06:44:26.107805 18512 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:26.107950 18512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:26.108327 18512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:26.111145 18512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "aaccd506de1c4a28b6d1b18b2cf7244b"
format_stamp: "Formatted at 2026-06-19 06:44:26 on dist-test-slave-5w2x"
server_key: "e5e3230634d4bea0fe7e543f3f32a34f"
server_key_iv: "8c36e2e4e7b8871222604e4b32b00770"
server_key_version: "encryptionkey@0"
I20260619 06:44:26.111905 18512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "aaccd506de1c4a28b6d1b18b2cf7244b"
format_stamp: "Formatted at 2026-06-19 06:44:26 on dist-test-slave-5w2x"
server_key: "e5e3230634d4bea0fe7e543f3f32a34f"
server_key_iv: "8c36e2e4e7b8871222604e4b32b00770"
server_key_version: "encryptionkey@0"
I20260619 06:44:26.117583 18512 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 06:44:26.121245 18527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:26.122534 18512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:44:26.122704 18512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "aaccd506de1c4a28b6d1b18b2cf7244b"
format_stamp: "Formatted at 2026-06-19 06:44:26 on dist-test-slave-5w2x"
server_key: "e5e3230634d4bea0fe7e543f3f32a34f"
server_key_iv: "8c36e2e4e7b8871222604e4b32b00770"
server_key_version: "encryptionkey@0"
I20260619 06:44:26.122903 18512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:26.140863 18512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:26.145110 18512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:26.145401 18512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:26.147893 18512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:26.148025 18512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:26.148185 18512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:26.148296 18512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Jun 19 06:44:26 dist-test-slave-5w2x krb5kdc[18396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851466, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:26.173578 18537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:26.149122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:43827 (local address 127.16.79.65:56945)
0619 06:44:26.149965 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:26.150008 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:26.152030 (+ 2022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:26.152859 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:26.152870 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:26.153600 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:26.154603 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:26.154634 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:26.156099 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:26.156114 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:26.158450 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:26.158466 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:26.159313 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:26.159322 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:26.159543 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:26.160360 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:26.160398 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:26.163329 (+ 2931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:26.167648 (+ 4319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:26.167661 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:26.167685 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:26.168627 (+ 942us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:26.169143 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:26.169148 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:26.169158 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:26.169455 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:26.170242 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:26.170252 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:26.170792 (+ 540us) client_negotiation.cc:769] Sending connection context
0619 06:44:26.171194 (+ 402us) client_negotiation.cc:240] Negotiation successful
0619 06:44:26.171810 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":347,"threads_started":1}
I20260619 06:44:26.174345 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:26.148521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:56945 (local address 127.16.79.126:43827)
0619 06:44:26.148990 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:26.148997 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:26.150134 (+ 1137us) server_negotiation.cc:407] Connection header received
0619 06:44:26.152360 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:26.152368 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:26.152482 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:26.152609 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:26.154826 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:26.155922 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:26.158647 (+ 2725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:26.159067 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:26.159204 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:26.163583 (+ 4379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:26.163592 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:26.163599 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:26.163649 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:26.167423 (+ 3774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:26.168828 (+ 1405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:26.168835 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:26.168840 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:26.169015 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:26.169611 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:26.169619 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:26.169624 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:26.170080 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:26.170245 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:26.173659 (+ 3414us) server_negotiation.cc:299] Negotiation successful
0619 06:44:26.173985 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":249,"threads_started":1}
I20260619 06:44:26.196012 18512 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:39209
I20260619 06:44:26.196108 18647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:39209 every 8 connection(s)
I20260619 06:44:26.197499 18512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:44:26.198652 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 18512
I20260619 06:44:26.198891 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:44:26.199772 16701 external_mini_cluster.cc:1428] Setting key cfc9092c1efe948ad4547e1515188965
I20260619 06:44:26.200845 18648 heartbeater.cc:347] Connected to a master server at 127.16.79.126:43827
I20260619 06:44:26.201081 18648 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:26.201792 18648 heartbeater.cc:511] Master 127.16.79.126:43827 requested a full tablet report, sending...
I20260619 06:44:26.202330 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:26.182453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:55795 (local address 127.16.79.126:43827)
0619 06:44:26.182704 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:26.182714 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:26.187621 (+ 4907us) server_negotiation.cc:407] Connection header received
0619 06:44:26.187770 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:26.187778 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:26.187909 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:26.188043 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:26.189817 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:26.191128 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:26.192725 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:26.193473 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:26.193626 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:26.196300 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:26.196307 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:26.196311 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:26.196352 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:26.199250 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:26.200199 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:26.200206 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:26.200209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:26.200400 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:26.200864 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:26.200869 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:26.200872 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:26.201165 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:26.201321 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:26.201755 (+ 434us) server_negotiation.cc:299] Negotiation successful
0619 06:44:26.201972 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 06:44:26.202461 18621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:26.181946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:43827 (local address 127.16.79.65:55795)
0619 06:44:26.187458 (+ 5512us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:26.187482 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:26.187641 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:26.188286 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:26.188291 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:26.189033 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:26.189555 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:26.189576 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:26.191293 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:26.191298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:26.192572 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:26.192585 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:26.193708 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:26.193717 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:26.193975 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:26.194711 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:26.194779 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:26.196028 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:26.199410 (+ 3382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:26.199417 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:26.199423 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:26.199980 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:26.200538 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:26.200545 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:26.200551 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:26.200712 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:26.201290 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:26.201295 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:26.201498 (+ 203us) client_negotiation.cc:769] Sending connection context
0619 06:44:26.201635 (+ 137us) client_negotiation.cc:240] Negotiation successful
0619 06:44:26.201991 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5345,"thread_start_us":5153,"threads_started":1}
I20260619 06:44:26.204051 18439 ts_manager.cc:194] Registered new tserver with Master: aaccd506de1c4a28b6d1b18b2cf7244b (127.16.79.65:39209)
I20260619 06:44:26.205482 18439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:56945
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:44:26.240446 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43827
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:26.573552 18652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:26.573837 18652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:26.573939 18652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:26.581362 18652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:26.581475 18652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:26.581614 18652 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 06:44:26.581672 18652 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:44:26.581784 18652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:44:26.591614 18652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.79.126:43827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:26.592962 18652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:26.594259 18652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:26.607650 18657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:26.607682 18658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:26.607764 18660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:26.614009 18652 server_base.cc:1034] running on GCE node
I20260619 06:44:26.614663 18652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:26.615645 18652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:26.616852 18652 hybrid_clock.cc:648] HybridClock initialized: now 1781851466616803 us; error 40 us; skew 500 ppm
I20260619 06:44:26.617189 18652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:26 dist-test-slave-5w2x krb5kdc[18396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851466, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:26.620504 18652 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:44:26.622141 18652 webserver.cc:460] Webserver started at http://127.16.79.66:39409/ using document root <none> and password file <none>
I20260619 06:44:26.622875 18652 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:26.623006 18652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:26.623338 18652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:26.626542 18652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "4ea302c03631485996ba1e7031de06be"
format_stamp: "Formatted at 2026-06-19 06:44:26 on dist-test-slave-5w2x"
server_key: "032bfaf8e885118ebb6ace9b1da9424b"
server_key_iv: "6c487b6c2616e55c47bed3d116902697"
server_key_version: "encryptionkey@0"
I20260619 06:44:26.627549 18652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "4ea302c03631485996ba1e7031de06be"
format_stamp: "Formatted at 2026-06-19 06:44:26 on dist-test-slave-5w2x"
server_key: "032bfaf8e885118ebb6ace9b1da9424b"
server_key_iv: "6c487b6c2616e55c47bed3d116902697"
server_key_version: "encryptionkey@0"
I20260619 06:44:26.633766 18652 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.001s
I20260619 06:44:26.637566 18667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:26.639061 18652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260619 06:44:26.639254 18652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "4ea302c03631485996ba1e7031de06be"
format_stamp: "Formatted at 2026-06-19 06:44:26 on dist-test-slave-5w2x"
server_key: "032bfaf8e885118ebb6ace9b1da9424b"
server_key_iv: "6c487b6c2616e55c47bed3d116902697"
server_key_version: "encryptionkey@0"
I20260619 06:44:26.639483 18652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:26.652498 18652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:26.656373 18652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:26.656665 18652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:26.659396 18652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:26.659531 18652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:26.659687 18652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:26.659761 18652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 06:44:26 dist-test-slave-5w2x krb5kdc[18396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851466, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:26.682850 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:26.659746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:54869 (local address 127.16.79.126:43827)
0619 06:44:26.659932 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:26.659937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:26.661484 (+ 1547us) server_negotiation.cc:407] Connection header received
0619 06:44:26.663679 (+ 2195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:26.663685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:26.663792 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:26.663932 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:26.666409 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:26.667502 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:26.669918 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:26.670366 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:26.670504 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:26.675726 (+ 5222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:26.675731 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:26.675735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:26.675772 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:26.679186 (+ 3414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:26.680236 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:26.680243 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:26.680247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:26.680413 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:26.681011 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:26.681017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:26.681019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:26.681422 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:26.681581 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:26.682277 (+ 696us) server_negotiation.cc:299] Negotiation successful
0619 06:44:26.682487 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 06:44:26.683881 18676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:26.660488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:43827 (local address 127.16.79.66:54869)
0619 06:44:26.661326 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:26.661363 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:26.663259 (+ 1896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:26.664276 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:26.664288 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:26.665129 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:26.666191 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:26.666223 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:26.667655 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:26.667660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:26.669757 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:26.669769 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:26.670602 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:26.670610 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:26.670999 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:26.671703 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:26.671728 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:26.675473 (+ 3745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:26.679352 (+ 3879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:26.679365 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:26.679385 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:26.680060 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:26.680576 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:26.680583 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:26.680588 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:26.680867 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:26.681554 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:26.681562 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:26.681915 (+ 353us) client_negotiation.cc:769] Sending connection context
0619 06:44:26.682168 (+ 253us) client_negotiation.cc:240] Negotiation successful
0619 06:44:26.682687 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":367,"threads_started":1}
I20260619 06:44:26.706599 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:26.690155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:60949 (local address 127.16.79.126:43827)
0619 06:44:26.690366 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:26.690374 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:26.691044 (+ 670us) server_negotiation.cc:407] Connection header received
0619 06:44:26.691228 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:26.691231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:26.691296 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:26.691373 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:26.692916 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:26.693624 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:26.696110 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:26.696669 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:26.696806 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:26.699789 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:26.699798 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:26.699805 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:26.699854 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:26.703752 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:26.704502 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:26.704509 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:26.704513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:26.704676 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:26.705044 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:26.705052 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:26.705057 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:26.705519 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:26.705681 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:26.706057 (+ 376us) server_negotiation.cc:299] Negotiation successful
0619 06:44:26.706271 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 06:44:26.706724 18744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:26.690160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:43827 (local address 127.16.79.66:60949)
0619 06:44:26.690912 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:26.690935 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:26.691109 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:26.691438 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:26.691446 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:26.692222 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:26.692739 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:26.692757 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:26.693779 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:26.693788 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:26.695960 (+ 2172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:26.695979 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:26.696927 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:26.696937 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:26.697192 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:26.697979 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:26.698014 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:26.699519 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:26.703888 (+ 4369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:26.703892 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:26.703895 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:26.704380 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:26.704796 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:26.704802 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:26.704806 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:26.704934 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:26.705643 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:26.705648 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:26.705833 (+ 185us) client_negotiation.cc:769] Sending connection context
0619 06:44:26.705973 (+ 140us) client_negotiation.cc:240] Negotiation successful
0619 06:44:26.706293 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":382,"threads_started":1}
I20260619 06:44:26.716797 18652 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:41289
I20260619 06:44:26.716826 18786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:41289 every 8 connection(s)
I20260619 06:44:26.717993 18652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:44:26.719986 18787 heartbeater.cc:347] Connected to a master server at 127.16.79.126:43827
I20260619 06:44:26.720234 18787 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:26.720835 18787 heartbeater.cc:511] Master 127.16.79.126:43827 requested a full tablet report, sending...
I20260619 06:44:26.721859 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 18652
I20260619 06:44:26.722038 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:44:26.722108 18439 ts_manager.cc:194] Registered new tserver with Master: 4ea302c03631485996ba1e7031de06be (127.16.79.66:41289)
I20260619 06:44:26.722563 16701 external_mini_cluster.cc:1428] Setting key 2901d0d2c2af3ba49140e4b137836861
I20260619 06:44:26.723161 18439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:54869
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:44:26.759989 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43827
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:33123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:27.073820 18791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:27.074093 18791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:27.074259 18791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:27.081563 18791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:27.081673 18791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:27.081866 18791 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 06:44:27.081944 18791 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:44:27.082067 18791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:44:27.092411 18791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:33123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.79.126:43827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:27.093693 18791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:27.095093 18791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:27.107787 18796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:27.107872 18797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:27.107913 18799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:27.108748 18791 server_base.cc:1034] running on GCE node
I20260619 06:44:27.109339 18791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:27.110258 18791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:27.111486 18791 hybrid_clock.cc:648] HybridClock initialized: now 1781851467111440 us; error 38 us; skew 500 ppm
I20260619 06:44:27.111811 18791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:27 dist-test-slave-5w2x krb5kdc[18396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851467, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:27.114825 18791 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:44:27.116289 18791 webserver.cc:460] Webserver started at http://127.16.79.67:43413/ using document root <none> and password file <none>
I20260619 06:44:27.116967 18791 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:27.117110 18791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:27.117419 18791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:27.121217 18791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "c05381a1ce2f43c0bdff51d620b89c2d"
format_stamp: "Formatted at 2026-06-19 06:44:27 on dist-test-slave-5w2x"
server_key: "bbe52302d6390aa3eb0a287d02866ae7"
server_key_iv: "f705d8058f46578d31ed38b7edf2a759"
server_key_version: "encryptionkey@0"
I20260619 06:44:27.122226 18791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "c05381a1ce2f43c0bdff51d620b89c2d"
format_stamp: "Formatted at 2026-06-19 06:44:27 on dist-test-slave-5w2x"
server_key: "bbe52302d6390aa3eb0a287d02866ae7"
server_key_iv: "f705d8058f46578d31ed38b7edf2a759"
server_key_version: "encryptionkey@0"
I20260619 06:44:27.128026 18791 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260619 06:44:27.131569 18806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:27.133067 18791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:44:27.133267 18791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "c05381a1ce2f43c0bdff51d620b89c2d"
format_stamp: "Formatted at 2026-06-19 06:44:27 on dist-test-slave-5w2x"
server_key: "bbe52302d6390aa3eb0a287d02866ae7"
server_key_iv: "f705d8058f46578d31ed38b7edf2a759"
server_key_version: "encryptionkey@0"
I20260619 06:44:27.133497 18791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:27.146281 18791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:27.150118 18791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:27.150410 18791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:27.153050 18791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:27.153206 18791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:27.153365 18791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:27.153473 18791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 06:44:27 dist-test-slave-5w2x krb5kdc[18396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851467, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:27.176961 18815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.154119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:43827 (local address 127.16.79.67:35233)
0619 06:44:27.154955 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:27.155003 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:27.156941 (+ 1938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:27.157729 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:27.157737 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:27.158496 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:27.159579 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.159607 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.161088 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.161097 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.163362 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.163374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.164207 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.164217 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.164497 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.165393 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:27.165446 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:27.168829 (+ 3383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:27.171896 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:27.171905 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:27.171922 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:27.172446 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:27.172906 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:27.172914 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:27.172920 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:27.173226 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:27.173899 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:27.173908 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:27.174272 (+ 364us) client_negotiation.cc:769] Sending connection context
0619 06:44:27.174660 (+ 388us) client_negotiation.cc:240] Negotiation successful
0619 06:44:27.175252 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":374,"threads_started":1}
I20260619 06:44:27.178141 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.153454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:35233 (local address 127.16.79.126:43827)
0619 06:44:27.153633 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:27.153642 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:27.155127 (+ 1485us) server_negotiation.cc:407] Connection header received
0619 06:44:27.157220 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:27.157229 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:27.157344 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:27.157453 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:27.159787 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.160934 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.163517 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.163927 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.164061 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.169062 (+ 5001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:27.169068 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:27.169073 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:27.169116 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:27.171748 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:27.172578 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:27.172587 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:27.172592 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:27.172763 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:27.173420 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:27.173427 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:27.173432 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:27.173753 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:27.173914 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:27.177598 (+ 3684us) server_negotiation.cc:299] Negotiation successful
0619 06:44:27.177828 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 06:44:27.199491 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.184928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:39861 (local address 127.16.79.126:43827)
0619 06:44:27.185126 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:27.185133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:27.186016 (+ 883us) server_negotiation.cc:407] Connection header received
0619 06:44:27.186232 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:27.186237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:27.186312 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:27.186412 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:27.187416 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.188125 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.190197 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.190799 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.190872 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.193466 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:27.193471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:27.193474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:27.193503 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:27.196906 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:27.197561 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:27.197567 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:27.197571 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:27.197699 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:27.198067 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:27.198072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:27.198075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:27.198348 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:27.198503 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:27.198877 (+ 374us) server_negotiation.cc:299] Negotiation successful
0619 06:44:27.199096 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 06:44:27.199549 18886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.185067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:43827 (local address 127.16.79.67:39861)
0619 06:44:27.185908 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:27.185927 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:27.186109 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:27.186438 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:27.186443 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:27.186990 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:27.187266 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.187283 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.188306 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.188311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.189988 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.190009 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.191226 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.191233 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.191380 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.192198 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:27.192224 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:27.193276 (+ 1052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:27.197042 (+ 3766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:27.197047 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:27.197051 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:27.197439 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:27.197813 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:27.197818 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:27.197822 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:27.197951 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:27.198462 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:27.198467 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:27.198644 (+ 177us) client_negotiation.cc:769] Sending connection context
0619 06:44:27.198790 (+ 146us) client_negotiation.cc:240] Negotiation successful
0619 06:44:27.199110 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":502,"threads_started":1}
I20260619 06:44:27.208321 18648 heartbeater.cc:503] Master 127.16.79.126:43827 was elected leader, sending a full tablet report...
I20260619 06:44:27.210299 18791 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:45515
I20260619 06:44:27.210322 18925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:45515 every 8 connection(s)
I20260619 06:44:27.211597 18791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:44:27.213263 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 18791
I20260619 06:44:27.213423 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:44:27.213757 18926 heartbeater.cc:347] Connected to a master server at 127.16.79.126:43827
I20260619 06:44:27.213975 16701 external_mini_cluster.cc:1428] Setting key 91cf0928fc132089c120025728ac40cd
I20260619 06:44:27.213989 18926 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:27.214723 18926 heartbeater.cc:511] Master 127.16.79.126:43827 requested a full tablet report, sending...
I20260619 06:44:27.216338 18440 ts_manager.cc:194] Registered new tserver with Master: c05381a1ce2f43c0bdff51d620b89c2d (127.16.79.67:45515)
I20260619 06:44:27.217185 18440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:35233
I20260619 06:44:27.229225 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 06:44:27 dist-test-slave-5w2x krb5kdc[18396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 06:44:27 dist-test-slave-5w2x krb5kdc[18396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:27.277014 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.261201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55726 (local address 127.16.79.126:43827)
0619 06:44:27.261584 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:27.261590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:27.261829 (+ 239us) server_negotiation.cc:407] Connection header received
0619 06:44:27.262011 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:27.262015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:27.262081 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:27.262220 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:27.264116 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.265242 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.266992 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.267577 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.267663 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.272392 (+ 4729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:27.272396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:27.272400 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:27.272437 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:27.274684 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:27.275330 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:27.275333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:27.275336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:27.275470 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:27.275778 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:27.275781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:27.275783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:27.276049 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:27.276297 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:27.276484 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 06:44:27.276691 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260619 06:44:27.281157 18440 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:44:27.288031 18440 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:44:27.312301 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.301295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46640 (local address 127.16.79.67:45515)
0619 06:44:27.302167 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:27.302175 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:27.303074 (+ 899us) server_negotiation.cc:407] Connection header received
0619 06:44:27.303694 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:27.303702 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:27.303972 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:27.304131 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:27.305214 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.306867 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.309460 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.310783 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.310918 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.311167 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:27.311700 (+ 533us) server_negotiation.cc:299] Negotiation successful
0619 06:44:27.311890 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":473,"threads_started":1}
I20260619 06:44:27.313047 18941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.300575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:39209 (local address 127.0.0.1:38412)
0619 06:44:27.302965 (+ 2390us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:27.302984 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:27.303461 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:27.304492 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:27.304499 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:27.304539 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:27.305128 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.305140 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.306996 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.307001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.309816 (+ 2815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.309832 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.311973 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.311980 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.312193 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.312261 (+ 68us) client_negotiation.cc:769] Sending connection context
0619 06:44:27.312385 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 06:44:27.312587 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2276,"mutex_wait_us":255,"spinlock_wait_cycles":545920,"thread_start_us":2079,"threads_started":1}
I20260619 06:44:27.313126 18939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.300575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:45515 (local address 127.0.0.1:46640)
0619 06:44:27.302748 (+ 2173us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:27.302945 (+ 197us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:27.303455 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:27.304377 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:27.304384 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:27.304411 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:27.304948 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.304965 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.307011 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.307016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.309292 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.309304 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.310975 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.310982 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.311410 (+ 428us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.311466 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 06:44:27.312578 (+ 1112us) client_negotiation.cc:240] Negotiation successful
0619 06:44:27.312684 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1897,"mutex_wait_us":106,"spinlock_wait_cycles":896,"thread_start_us":2252,"threads_started":1}
I20260619 06:44:27.313262 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.304075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54236 (local address 127.16.79.66:41289)
0619 06:44:27.304833 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:27.304843 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:27.305420 (+ 577us) server_negotiation.cc:407] Connection header received
0619 06:44:27.305560 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:27.305567 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:27.305799 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:27.305974 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:27.306547 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.307965 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.310347 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.311645 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.311762 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.311940 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:27.312790 (+ 850us) server_negotiation.cc:299] Negotiation successful
0619 06:44:27.312961 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":460,"threads_started":1}
I20260619 06:44:27.313529 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.300644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38412 (local address 127.16.79.65:39209)
0619 06:44:27.301340 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:27.301348 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:27.303075 (+ 1727us) server_negotiation.cc:407] Connection header received
0619 06:44:27.303705 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:27.303716 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:27.303974 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:27.304130 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:27.305335 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.306675 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.309991 (+ 3316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.311472 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.311610 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.311830 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:27.312676 (+ 846us) server_negotiation.cc:299] Negotiation successful
0619 06:44:27.312854 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"spinlock_wait_cycles":1280,"thread_start_us":513,"threads_started":1}
I20260619 06:44:27.314507 18943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.303598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:41289 (local address 127.0.0.1:54236)
0619 06:44:27.305292 (+ 1694us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:27.305312 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:27.305428 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:27.306031 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:27.306036 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:27.306058 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:27.306388 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.306399 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.308119 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.308124 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.310209 (+ 2085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.310219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.312105 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.312111 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.312432 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.312470 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 06:44:27.312556 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 06:44:27.312684 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1565,"mutex_wait_us":95,"thread_start_us":1417,"threads_started":1}
I20260619 06:44:27.316995 18578 tablet_service.cc:1467] Processing CreateTablet for tablet 0a9c2cb33ad345678f1ffb71fcbd7303 (DEFAULT_TABLE table=test-table [id=8bb38542fbc44dc285a28c8ded57f1fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:27.318323 18578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a9c2cb33ad345678f1ffb71fcbd7303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:27.317888 18717 tablet_service.cc:1467] Processing CreateTablet for tablet 0a9c2cb33ad345678f1ffb71fcbd7303 (DEFAULT_TABLE table=test-table [id=8bb38542fbc44dc285a28c8ded57f1fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:27.318617 18856 tablet_service.cc:1467] Processing CreateTablet for tablet 0a9c2cb33ad345678f1ffb71fcbd7303 (DEFAULT_TABLE table=test-table [id=8bb38542fbc44dc285a28c8ded57f1fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:27.319235 18717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a9c2cb33ad345678f1ffb71fcbd7303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:27.319936 18856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a9c2cb33ad345678f1ffb71fcbd7303. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:27.331743 18945 tablet_bootstrap.cc:492] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b: Bootstrap starting.
I20260619 06:44:27.334019 18947 tablet_bootstrap.cc:492] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d: Bootstrap starting.
I20260619 06:44:27.335654 18946 tablet_bootstrap.cc:492] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be: Bootstrap starting.
I20260619 06:44:27.335696 18945 tablet_bootstrap.cc:654] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:27.337150 18945 log.cc:824] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:27.337608 18947 tablet_bootstrap.cc:654] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:27.338797 18947 log.cc:824] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:27.339304 18946 tablet_bootstrap.cc:654] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:27.340471 18946 log.cc:824] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:27.341476 18945 tablet_bootstrap.cc:492] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b: No bootstrap required, opened a new log
I20260619 06:44:27.341770 18945 ts_tablet_manager.cc:1397] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260619 06:44:27.342686 18947 tablet_bootstrap.cc:492] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d: No bootstrap required, opened a new log
I20260619 06:44:27.342938 18947 ts_tablet_manager.cc:1397] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260619 06:44:27.346175 18946 tablet_bootstrap.cc:492] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be: No bootstrap required, opened a new log
I20260619 06:44:27.346450 18946 ts_tablet_manager.cc:1397] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260619 06:44:27.353051 18945 raft_consensus.cc:348] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } }
I20260619 06:44:27.353538 18945 raft_consensus.cc:374] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:27.353291 18947 raft_consensus.cc:348] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } }
I20260619 06:44:27.353669 18947 raft_consensus.cc:374] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:27.353643 18945 raft_consensus.cc:729] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaccd506de1c4a28b6d1b18b2cf7244b, State: Initialized, Role: FOLLOWER
I20260619 06:44:27.353773 18947 raft_consensus.cc:729] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c05381a1ce2f43c0bdff51d620b89c2d, State: Initialized, Role: FOLLOWER
I20260619 06:44:27.353662 18946 raft_consensus.cc:348] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } }
I20260619 06:44:27.354148 18946 raft_consensus.cc:374] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:27.354277 18946 raft_consensus.cc:729] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ea302c03631485996ba1e7031de06be, State: Initialized, Role: FOLLOWER
I20260619 06:44:27.354354 18947 consensus_queue.cc:260] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } }
I20260619 06:44:27.354327 18945 consensus_queue.cc:260] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } }
I20260619 06:44:27.354874 18946 consensus_queue.cc:260] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } }
I20260619 06:44:27.356031 18926 heartbeater.cc:503] Master 127.16.79.126:43827 was elected leader, sending a full tablet report...
I20260619 06:44:27.356295 18945 ts_tablet_manager.cc:1428] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b: Time spent starting tablet: real 0.014s user 0.014s sys 0.002s
I20260619 06:44:27.357540 18947 ts_tablet_manager.cc:1428] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260619 06:44:27.363739 18787 heartbeater.cc:503] Master 127.16.79.126:43827 was elected leader, sending a full tablet report...
I20260619 06:44:27.364689 18946 ts_tablet_manager.cc:1428] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be: Time spent starting tablet: real 0.018s user 0.012s sys 0.005s
W20260619 06:44:27.451232 18649 tablet.cc:2369] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:44:27.461809 18951 raft_consensus.cc:482] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:44:27.462075 18951 raft_consensus.cc:504] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } }
W20260619 06:44:27.463127 18927 tablet.cc:2369] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:44:27.463719 18951 leader_election.cc:288] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c05381a1ce2f43c0bdff51d620b89c2d (127.16.79.67:45515), 4ea302c03631485996ba1e7031de06be (127.16.79.66:41289)
W20260619 06:44:27.469995 18788 tablet.cc:2369] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:44:27.470641 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.464119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:44773 (local address 127.16.79.67:45515)
0619 06:44:27.464330 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:27.464338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:27.465186 (+ 848us) server_negotiation.cc:407] Connection header received
0619 06:44:27.465440 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:27.465444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:27.465500 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:27.465599 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:27.466237 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.466921 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.468893 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.469624 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.469802 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.469879 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:27.470272 (+ 393us) server_negotiation.cc:299] Negotiation successful
0619 06:44:27.470392 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 06:44:27.470796 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.464074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:41209 (local address 127.16.79.66:41289)
0619 06:44:27.464305 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:27.464311 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:27.465200 (+ 889us) server_negotiation.cc:407] Connection header received
0619 06:44:27.465407 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:27.465413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:27.465482 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:27.465547 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:27.466161 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.466891 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.468924 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.469669 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.469836 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.469906 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:27.470424 (+ 518us) server_negotiation.cc:299] Negotiation successful
0619 06:44:27.470552 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260619 06:44:27.470938 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.464094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:45515 (local address 127.16.79.65:44773)
0619 06:44:27.465069 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:27.465085 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:27.465270 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:27.465628 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:27.465641 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:27.465664 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:27.466069 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.466100 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.467075 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.467079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.468742 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.468753 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.469816 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.469823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.469990 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.470013 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 06:44:27.470247 (+ 234us) client_negotiation.cc:240] Negotiation successful
0619 06:44:27.470373 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"mutex_wait_us":27,"spinlock_wait_cycles":9344,"thread_start_us":438,"threads_started":1}
I20260619 06:44:27.471370 18955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.464331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:41289 (local address 127.16.79.65:41209)
0619 06:44:27.465078 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:27.465092 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:27.465274 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:27.465590 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:27.465595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:27.465618 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:27.466005 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.466014 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.467016 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.467025 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.468747 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.468764 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.470050 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.470056 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.470223 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.470253 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 06:44:27.470966 (+ 713us) client_negotiation.cc:240] Negotiation successful
0619 06:44:27.471089 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":87,"spinlock_wait_cycles":11136,"thread_start_us":374,"threads_started":1}
I20260619 06:44:27.471894 18876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a9c2cb33ad345678f1ffb71fcbd7303" candidate_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" is_pre_election: true
I20260619 06:44:27.472326 18738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a9c2cb33ad345678f1ffb71fcbd7303" candidate_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ea302c03631485996ba1e7031de06be" is_pre_election: true
I20260619 06:44:27.472420 18876 raft_consensus.cc:2454] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaccd506de1c4a28b6d1b18b2cf7244b in term 0.
I20260619 06:44:27.472746 18738 raft_consensus.cc:2454] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaccd506de1c4a28b6d1b18b2cf7244b in term 0.
I20260619 06:44:27.473053 18528 leader_election.cc:302] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaccd506de1c4a28b6d1b18b2cf7244b, c05381a1ce2f43c0bdff51d620b89c2d; no voters:
I20260619 06:44:27.473495 18951 raft_consensus.cc:2781] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:44:27.473634 18951 raft_consensus.cc:482] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:44:27.473760 18951 raft_consensus.cc:3037] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:27.475831 18951 raft_consensus.cc:504] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } }
I20260619 06:44:27.476552 18951 leader_election.cc:288] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [CANDIDATE]: Term 1 election: Requested vote from peers c05381a1ce2f43c0bdff51d620b89c2d (127.16.79.67:45515), 4ea302c03631485996ba1e7031de06be (127.16.79.66:41289)
I20260619 06:44:27.476939 18876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a9c2cb33ad345678f1ffb71fcbd7303" candidate_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c05381a1ce2f43c0bdff51d620b89c2d"
I20260619 06:44:27.476956 18738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a9c2cb33ad345678f1ffb71fcbd7303" candidate_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ea302c03631485996ba1e7031de06be"
I20260619 06:44:27.477119 18876 raft_consensus.cc:3037] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:27.477152 18738 raft_consensus.cc:3037] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:27.479231 18876 raft_consensus.cc:2454] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaccd506de1c4a28b6d1b18b2cf7244b in term 1.
I20260619 06:44:27.479242 18738 raft_consensus.cc:2454] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaccd506de1c4a28b6d1b18b2cf7244b in term 1.
I20260619 06:44:27.479653 18528 leader_election.cc:302] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaccd506de1c4a28b6d1b18b2cf7244b, c05381a1ce2f43c0bdff51d620b89c2d; no voters:
I20260619 06:44:27.479987 18951 raft_consensus.cc:2781] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:27.480490 18951 raft_consensus.cc:686] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 LEADER]: Becoming Leader. State: Replica: aaccd506de1c4a28b6d1b18b2cf7244b, State: Running, Role: LEADER
I20260619 06:44:27.480965 18951 consensus_queue.cc:237] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } }
I20260619 06:44:27.485615 18440 catalog_manager.cc:5495] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b reported cstate change: term changed from 0 to 1, leader changed from <none> to aaccd506de1c4a28b6d1b18b2cf7244b (127.16.79.65). New cstate: current_term: 1 leader_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } health_report { overall_health: UNKNOWN } } }
Jun 19 06:44:27 dist-test-slave-5w2x krb5kdc[18396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851465, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:27.539682 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.526677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55736 (local address 127.16.79.126:43827)
0619 06:44:27.526952 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:27.526959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:27.527550 (+ 591us) server_negotiation.cc:407] Connection header received
0619 06:44:27.527680 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:27.527684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:27.527755 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:27.527854 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:27.529003 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.529694 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.530880 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.531286 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.531377 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.534640 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:27.534645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:27.534648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:27.534673 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:27.537083 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:27.537763 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:27.537768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:27.537771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:27.537936 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:27.538279 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:27.538284 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:27.538287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:27.538545 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:27.538693 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:27.539084 (+ 391us) server_negotiation.cc:299] Negotiation successful
0619 06:44:27.539369 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":83}
I20260619 06:44:27.539716 18963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.526798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:43827 (local address 127.0.0.1:55736)
0619 06:44:27.527445 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:27.527464 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:27.527585 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:27.527924 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:27.527933 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:27.528612 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:27.528867 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.528876 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.529807 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.529811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.530745 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.530764 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.531433 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.531437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.531548 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.532288 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:27.532305 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:27.534462 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:27.537216 (+ 2754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:27.537227 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:27.537244 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:27.537594 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:27.538039 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:27.538046 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:27.538049 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:27.538181 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:27.538671 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:27.538678 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:27.538839 (+ 161us) client_negotiation.cc:769] Sending connection context
0619 06:44:27.538993 (+ 154us) client_negotiation.cc:240] Negotiation successful
0619 06:44:27.539256 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":361,"threads_started":1}
I20260619 06:44:27.544721 18440 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:55736:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 06:44:27.545648 18440 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:44:27.551868 18578 tablet_service.cc:1467] Processing CreateTablet for tablet b420fd4da0ed463587fad58c2eac7bbf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=90c164a0a63f427aab54110140412310]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:44:27.551942 18717 tablet_service.cc:1467] Processing CreateTablet for tablet b420fd4da0ed463587fad58c2eac7bbf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=90c164a0a63f427aab54110140412310]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:44:27.552248 18856 tablet_service.cc:1467] Processing CreateTablet for tablet b420fd4da0ed463587fad58c2eac7bbf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=90c164a0a63f427aab54110140412310]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:44:27.552883 18578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b420fd4da0ed463587fad58c2eac7bbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:27.552897 18717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b420fd4da0ed463587fad58c2eac7bbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:27.552966 18856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b420fd4da0ed463587fad58c2eac7bbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:27.558725 18945 tablet_bootstrap.cc:492] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b: Bootstrap starting.
I20260619 06:44:27.559899 18947 tablet_bootstrap.cc:492] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d: Bootstrap starting.
I20260619 06:44:27.560228 18945 tablet_bootstrap.cc:654] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:27.560339 18946 tablet_bootstrap.cc:492] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be: Bootstrap starting.
I20260619 06:44:27.561563 18947 tablet_bootstrap.cc:654] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:27.561710 18946 tablet_bootstrap.cc:654] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:27.562523 18945 tablet_bootstrap.cc:492] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b: No bootstrap required, opened a new log
I20260619 06:44:27.562685 18945 ts_tablet_manager.cc:1397] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260619 06:44:27.563680 18945 raft_consensus.cc:348] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:27.563989 18945 raft_consensus.cc:374] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:27.564101 18945 raft_consensus.cc:729] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaccd506de1c4a28b6d1b18b2cf7244b, State: Initialized, Role: FOLLOWER
I20260619 06:44:27.564415 18945 consensus_queue.cc:260] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:27.564952 18951 tablet_replica.cc:439] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } }
I20260619 06:44:27.565260 18945 ts_tablet_manager.cc:1428] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 06:44:27.565301 18951 tablet_replica.cc:442] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:44:27.567087 18946 tablet_bootstrap.cc:492] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be: No bootstrap required, opened a new log
I20260619 06:44:27.567233 18946 ts_tablet_manager.cc:1397] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260619 06:44:27.568004 18946 raft_consensus.cc:348] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:27.568298 18946 raft_consensus.cc:374] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:27.568403 18946 raft_consensus.cc:729] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ea302c03631485996ba1e7031de06be, State: Initialized, Role: FOLLOWER
I20260619 06:44:27.568663 18946 consensus_queue.cc:260] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:27.569135 18953 tablet_replica.cc:439] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } }
I20260619 06:44:27.569460 18953 tablet_replica.cc:442] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:44:27.569937 18946 ts_tablet_manager.cc:1428] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 06:44:27.576115 18947 tablet_bootstrap.cc:492] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d: No bootstrap required, opened a new log
I20260619 06:44:27.576300 18947 ts_tablet_manager.cc:1397] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.009s
I20260619 06:44:27.576936 18947 raft_consensus.cc:348] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:27.577116 18947 raft_consensus.cc:374] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:27.577173 18947 raft_consensus.cc:729] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c05381a1ce2f43c0bdff51d620b89c2d, State: Initialized, Role: FOLLOWER
I20260619 06:44:27.577389 18947 consensus_queue.cc:260] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:27.577936 18947 ts_tablet_manager.cc:1428] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 06:44:27.577863 18952 tablet_replica.cc:439] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } }
I20260619 06:44:27.578272 18952 tablet_replica.cc:442] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:44:27.607215 18953 raft_consensus.cc:482] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:44:27.607488 18953 raft_consensus.cc:504] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:27.609117 18953 leader_election.cc:288] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aaccd506de1c4a28b6d1b18b2cf7244b (127.16.79.65:39209), c05381a1ce2f43c0bdff51d620b89c2d (127.16.79.67:45515)
I20260619 06:44:27.615294 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.609487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:51829 (local address 127.16.79.65:39209)
0619 06:44:27.609719 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:27.609726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:27.610194 (+ 468us) server_negotiation.cc:407] Connection header received
0619 06:44:27.610404 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:27.610408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:27.610465 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:27.610562 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:27.611102 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.611813 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.613660 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.614397 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.614581 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.614657 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:27.614934 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 06:44:27.615056 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 06:44:27.615396 18967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.609382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:39209 (local address 127.16.79.66:51829)
0619 06:44:27.610067 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:27.610089 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:27.610237 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:27.610576 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:27.610581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:27.610605 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:27.610929 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.610941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.611939 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.611946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.613494 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.613507 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.614607 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.614614 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.614776 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.614800 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 06:44:27.614908 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 06:44:27.615035 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":346,"threads_started":1}
I20260619 06:44:27.615520 18942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.609577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:44035 (local address 127.16.79.67:45515)
0619 06:44:27.609738 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:27.609745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:27.610320 (+ 575us) server_negotiation.cc:407] Connection header received
0619 06:44:27.610512 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:27.610516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:27.610573 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:27.610669 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:27.611298 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.612041 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.613735 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.614570 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.614746 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.614819 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:27.615101 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 06:44:27.615230 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 06:44:27.615960 18968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.609478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:45515 (local address 127.16.79.66:44035)
0619 06:44:27.610203 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:27.610221 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:27.610377 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:27.610690 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:27.610698 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:27.610730 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:27.611145 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.611153 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.612203 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.612209 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.613584 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.613597 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.614783 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.614790 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.614946 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.614968 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 06:44:27.615067 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 06:44:27.615267 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":423,"threads_started":1}
I20260619 06:44:27.616426 18598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b420fd4da0ed463587fad58c2eac7bbf" candidate_uuid: "4ea302c03631485996ba1e7031de06be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" is_pre_election: true
I20260619 06:44:27.616631 18876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b420fd4da0ed463587fad58c2eac7bbf" candidate_uuid: "4ea302c03631485996ba1e7031de06be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" is_pre_election: true
I20260619 06:44:27.616858 18876 raft_consensus.cc:2454] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ea302c03631485996ba1e7031de06be in term 0.
I20260619 06:44:27.616894 18598 raft_consensus.cc:2454] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ea302c03631485996ba1e7031de06be in term 0.
I20260619 06:44:27.617378 18668 leader_election.cc:302] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ea302c03631485996ba1e7031de06be, c05381a1ce2f43c0bdff51d620b89c2d; no voters:
I20260619 06:44:27.617776 18953 raft_consensus.cc:2781] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:44:27.617918 18953 raft_consensus.cc:482] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:44:27.618016 18953 raft_consensus.cc:3037] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:27.620311 18953 raft_consensus.cc:504] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:27.620913 18953 leader_election.cc:288] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [CANDIDATE]: Term 1 election: Requested vote from peers aaccd506de1c4a28b6d1b18b2cf7244b (127.16.79.65:39209), c05381a1ce2f43c0bdff51d620b89c2d (127.16.79.67:45515)
I20260619 06:44:27.621390 18598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b420fd4da0ed463587fad58c2eac7bbf" candidate_uuid: "4ea302c03631485996ba1e7031de06be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b"
I20260619 06:44:27.621373 18876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b420fd4da0ed463587fad58c2eac7bbf" candidate_uuid: "4ea302c03631485996ba1e7031de06be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c05381a1ce2f43c0bdff51d620b89c2d"
I20260619 06:44:27.621577 18598 raft_consensus.cc:3037] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:27.621595 18876 raft_consensus.cc:3037] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:27.624622 18876 raft_consensus.cc:2454] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ea302c03631485996ba1e7031de06be in term 1.
I20260619 06:44:27.624637 18598 raft_consensus.cc:2454] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ea302c03631485996ba1e7031de06be in term 1.
I20260619 06:44:27.625101 18668 leader_election.cc:302] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4ea302c03631485996ba1e7031de06be, aaccd506de1c4a28b6d1b18b2cf7244b, c05381a1ce2f43c0bdff51d620b89c2d; no voters:
I20260619 06:44:27.625449 18953 raft_consensus.cc:2781] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:27.625900 18953 raft_consensus.cc:686] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [term 1 LEADER]: Becoming Leader. State: Replica: 4ea302c03631485996ba1e7031de06be, State: Running, Role: LEADER
I20260619 06:44:27.626466 18953 consensus_queue.cc:237] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:27.630038 18969 tablet_replica.cc:439] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be: TxnStatusTablet state changed. Reason: New leader 4ea302c03631485996ba1e7031de06be. Latest consensus state: current_term: 1 leader_uuid: "4ea302c03631485996ba1e7031de06be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } }
I20260619 06:44:27.630440 18969 tablet_replica.cc:442] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:44:27.631153 18971 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 06:44:27.631431 18971 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 06:44:27.631644 18439 catalog_manager.cc:5495] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ea302c03631485996ba1e7031de06be (127.16.79.66). New cstate: current_term: 1 leader_uuid: "4ea302c03631485996ba1e7031de06be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } health_report { overall_health: UNKNOWN } } }
I20260619 06:44:27.649804 18963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.644243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:41289 (local address 127.0.0.1:54248)
0619 06:44:27.644486 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:27.644505 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:27.644687 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:27.645139 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:27.645143 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:27.645155 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:44:27.645364 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.645374 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.646418 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.646426 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.647404 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.647415 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.648174 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.648180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.648378 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.648432 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:44:27.649256 (+ 824us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:44:27.649285 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 06:44:27.649355 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 06:44:27.649517 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260619 06:44:27.649837 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.644310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54248 (local address 127.16.79.66:41289)
0619 06:44:27.644474 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:27.644483 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:27.644587 (+ 104us) server_negotiation.cc:407] Connection header received
0619 06:44:27.644881 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:27.644886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:27.644981 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:27.645126 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:44:27.645490 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.646286 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.647549 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.647992 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.648121 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.648633 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:44:27.649146 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:44:27.649312 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:27.649372 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 06:44:27.649457 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 06:44:27.657920 18598 raft_consensus.cc:1261] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 FOLLOWER]: Refusing update from remote peer 4ea302c03631485996ba1e7031de06be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:27.658037 18876 raft_consensus.cc:1261] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Refusing update from remote peer 4ea302c03631485996ba1e7031de06be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:27.658907 18969 consensus_queue.cc:1035] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:27.659219 18953 consensus_queue.cc:1035] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be [LEADER]: Connected to new peer: Peer: permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:27.669605 18951 tablet_replica.cc:439] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b: TxnStatusTablet state changed. Reason: New leader 4ea302c03631485996ba1e7031de06be. Latest consensus state: current_term: 1 leader_uuid: "4ea302c03631485996ba1e7031de06be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } }
I20260619 06:44:27.670090 18951 tablet_replica.cc:442] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:44:27.672046 18974 mvcc.cc:205] Tried to move back new op lower bound from 7298463611517661184 to 7298463611402465280. Current Snapshot: MvccSnapshot[applied={T|T < 7298463611517661184}]
I20260619 06:44:27.673666 18953 tablet_replica.cc:439] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ea302c03631485996ba1e7031de06be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } }
I20260619 06:44:27.673950 18953 tablet_replica.cc:442] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:44:27.675077 18969 tablet_replica.cc:439] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ea302c03631485996ba1e7031de06be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } }
I20260619 06:44:27.675338 18969 tablet_replica.cc:442] T b420fd4da0ed463587fad58c2eac7bbf P 4ea302c03631485996ba1e7031de06be: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:44:27.676707 18952 tablet_replica.cc:439] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d: TxnStatusTablet state changed. Reason: New leader 4ea302c03631485996ba1e7031de06be. Latest consensus state: current_term: 1 leader_uuid: "4ea302c03631485996ba1e7031de06be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } }
I20260619 06:44:27.676970 18952 tablet_replica.cc:442] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:44:27.681118 18951 tablet_replica.cc:439] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ea302c03631485996ba1e7031de06be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } }
I20260619 06:44:27.681442 18951 tablet_replica.cc:442] T b420fd4da0ed463587fad58c2eac7bbf P aaccd506de1c4a28b6d1b18b2cf7244b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:44:27.684953 18952 tablet_replica.cc:439] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ea302c03631485996ba1e7031de06be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } } }
I20260619 06:44:27.685196 18952 tablet_replica.cc:442] T b420fd4da0ed463587fad58c2eac7bbf P c05381a1ce2f43c0bdff51d620b89c2d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:44:27.694160 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.687912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38418 (local address 127.16.79.65:39209)
0619 06:44:27.688081 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:27.688086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:27.688370 (+ 284us) server_negotiation.cc:407] Connection header received
0619 06:44:27.688532 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:27.688536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:27.688618 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:27.688736 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:44:27.689145 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.689878 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.691853 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.692329 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.692423 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.692970 (+ 547us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:44:27.693496 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:44:27.693600 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:27.693750 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 06:44:27.693828 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 06:44:27.712107 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.705822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:60807 (local address 127.16.79.66:41289)
0619 06:44:27.706053 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:27.706062 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:27.706615 (+ 553us) server_negotiation.cc:407] Connection header received
0619 06:44:27.706798 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:27.706801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:27.706882 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:27.706965 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:44:27.707548 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.708378 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.710405 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.710843 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.710953 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.711511 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:44:27.711610 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:44:27.711673 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:27.711829 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 06:44:27.711890 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 06:44:27.712378 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.705812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:41289 (local address 127.16.79.65:60807)
0619 06:44:27.706500 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:27.706520 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:27.706670 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:27.707006 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:27.707011 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:27.707024 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:44:27.707376 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.707389 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.708499 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.708505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.710271 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.710286 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.711030 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.711037 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.711211 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.711294 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:44:27.711701 (+ 407us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:44:27.711713 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 06:44:27.711796 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 06:44:27.711945 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":345,"threads_started":1}
I20260619 06:44:27.724221 18876 raft_consensus.cc:1261] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Refusing update from remote peer aaccd506de1c4a28b6d1b18b2cf7244b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:27.724449 18738 raft_consensus.cc:1261] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Refusing update from remote peer aaccd506de1c4a28b6d1b18b2cf7244b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:27.724961 18956 consensus_queue.cc:1035] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:27.725404 18951 consensus_queue.cc:1035] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:27.741667 18976 mvcc.cc:205] Tried to move back new op lower bound from 7298463611792388096 to 7298463610805559296. Current Snapshot: MvccSnapshot[applied={T|T < 7298463611792388096}]
I20260619 06:44:27.773414 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.765889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:45925 (local address 127.16.79.65:39209)
0619 06:44:27.766089 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:27.766096 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:27.766901 (+ 805us) server_negotiation.cc:407] Connection header received
0619 06:44:27.767095 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:27.767100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:27.767192 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:27.767282 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:44:27.768126 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.769270 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.771516 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.771986 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.772165 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.772649 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:44:27.772753 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:44:27.772859 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:27.772997 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 06:44:27.773080 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:44:27.773660 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:27.765922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:39209 (local address 127.16.79.66:45925)
0619 06:44:27.766786 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:27.766805 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:27.766957 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:27.767420 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:27.767425 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:27.767441 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:44:27.767943 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.767966 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.769456 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.769461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.771381 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:27.771395 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:27.772233 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:27.772240 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:27.772387 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:27.772444 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:44:27.772860 (+ 416us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:44:27.772880 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 06:44:27.772976 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 06:44:27.773119 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":483,"threads_started":1}
I20260619 06:44:27.811061 18439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:44:27.811357 18439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:44:27.815336 18439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a: Sending DeleteTablet for 3 replicas of tablet 0a9c2cb33ad345678f1ffb71fcbd7303
I20260619 06:44:27.817056 18856 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a9c2cb33ad345678f1ffb71fcbd7303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:27 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:46640
I20260619 06:44:27.817324 18578 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a9c2cb33ad345678f1ffb71fcbd7303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:27 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:38412
I20260619 06:44:27.818315 18717 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a9c2cb33ad345678f1ffb71fcbd7303 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:27 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:54236
I20260619 06:44:27.819016 18439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:44:27.819890 18439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:44:27.824123 18995 tablet_replica.cc:330] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be: stopping tablet replica
I20260619 06:44:27.824644 18995 raft_consensus.cc:2229] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:44:27.825023 18995 raft_consensus.cc:2258] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:27.827435 18995 ts_tablet_manager.cc:1905] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:27.831130 18577 tablet_service.cc:1467] Processing CreateTablet for tablet 241a22e17b8f49cc89d71b10fac1d7fa (DEFAULT_TABLE table=test-table [id=21a598ab744e4b8a8b388ef61be87621]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:27.831688 18577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241a22e17b8f49cc89d71b10fac1d7fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:27.836859 18855 tablet_service.cc:1467] Processing CreateTablet for tablet 241a22e17b8f49cc89d71b10fac1d7fa (DEFAULT_TABLE table=test-table [id=21a598ab744e4b8a8b388ef61be87621]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:27.837414 18855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241a22e17b8f49cc89d71b10fac1d7fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:27.837693 18717 tablet_service.cc:1467] Processing CreateTablet for tablet 241a22e17b8f49cc89d71b10fac1d7fa (DEFAULT_TABLE table=test-table [id=21a598ab744e4b8a8b388ef61be87621]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:27.838106 18717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241a22e17b8f49cc89d71b10fac1d7fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:27.843035 18945 tablet_bootstrap.cc:492] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b: Bootstrap starting.
I20260619 06:44:27.844884 18945 tablet_bootstrap.cc:654] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:27.846089 18947 tablet_bootstrap.cc:492] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d: Bootstrap starting.
I20260619 06:44:27.847807 18947 tablet_bootstrap.cc:654] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:27.860977 18946 tablet_bootstrap.cc:492] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be: Bootstrap starting.
I20260619 06:44:27.863058 18946 tablet_bootstrap.cc:654] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:27.868171 18947 tablet_bootstrap.cc:492] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d: No bootstrap required, opened a new log
I20260619 06:44:27.868444 18947 ts_tablet_manager.cc:1397] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d: Time spent bootstrapping tablet: real 0.022s user 0.000s sys 0.010s
I20260619 06:44:27.869227 18945 tablet_bootstrap.cc:492] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b: No bootstrap required, opened a new log
I20260619 06:44:27.869449 18945 ts_tablet_manager.cc:1397] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b: Time spent bootstrapping tablet: real 0.026s user 0.006s sys 0.005s
I20260619 06:44:27.869422 18947 raft_consensus.cc:348] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:27.869856 18947 raft_consensus.cc:374] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:27.869987 18947 raft_consensus.cc:729] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c05381a1ce2f43c0bdff51d620b89c2d, State: Initialized, Role: FOLLOWER
I20260619 06:44:27.870358 18947 consensus_queue.cc:260] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:27.870420 18945 raft_consensus.cc:348] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:27.870776 18945 raft_consensus.cc:374] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:27.870875 18945 raft_consensus.cc:729] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaccd506de1c4a28b6d1b18b2cf7244b, State: Initialized, Role: FOLLOWER
I20260619 06:44:27.871230 18945 consensus_queue.cc:260] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:27.873525 18947 ts_tablet_manager.cc:1428] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260619 06:44:27.875598 18945 ts_tablet_manager.cc:1428] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260619 06:44:27.875886 18994 tablet_replica.cc:330] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b: stopping tablet replica
I20260619 06:44:27.876480 18994 raft_consensus.cc:2229] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:44:27.877285 18994 raft_consensus.cc:2258] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:27.877513 18995 ts_tablet_manager.cc:1918] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:44:27.877712 18995 log.cc:1196] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/0a9c2cb33ad345678f1ffb71fcbd7303
I20260619 06:44:27.878283 18995 ts_tablet_manager.cc:1939] T 0a9c2cb33ad345678f1ffb71fcbd7303 P 4ea302c03631485996ba1e7031de06be: Deleting consensus metadata
I20260619 06:44:27.880081 18428 catalog_manager.cc:4841] TS 4ea302c03631485996ba1e7031de06be (127.16.79.66:41289): tablet 0a9c2cb33ad345678f1ffb71fcbd7303 (table test-table [id=8bb38542fbc44dc285a28c8ded57f1fc]) successfully deleted
I20260619 06:44:27.880734 18994 ts_tablet_manager.cc:1905] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:27.880964 18993 tablet_replica.cc:330] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d: stopping tablet replica
I20260619 06:44:27.881410 18993 raft_consensus.cc:2229] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:44:27.881474 18946 tablet_bootstrap.cc:492] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be: No bootstrap required, opened a new log
I20260619 06:44:27.881652 18946 ts_tablet_manager.cc:1397] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.004s
I20260619 06:44:27.881762 18993 raft_consensus.cc:2258] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:27.882453 18946 raft_consensus.cc:348] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:27.882705 18946 raft_consensus.cc:374] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:27.882788 18946 raft_consensus.cc:729] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ea302c03631485996ba1e7031de06be, State: Initialized, Role: FOLLOWER
I20260619 06:44:27.883062 18946 consensus_queue.cc:260] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:27.883744 18946 ts_tablet_manager.cc:1428] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260619 06:44:27.888314 18993 ts_tablet_manager.cc:1905] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:27.889514 18994 ts_tablet_manager.cc:1918] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:44:27.889688 18994 log.cc:1196] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/0a9c2cb33ad345678f1ffb71fcbd7303
I20260619 06:44:27.890261 18994 ts_tablet_manager.cc:1939] T 0a9c2cb33ad345678f1ffb71fcbd7303 P aaccd506de1c4a28b6d1b18b2cf7244b: Deleting consensus metadata
I20260619 06:44:27.892045 18428 catalog_manager.cc:4841] TS aaccd506de1c4a28b6d1b18b2cf7244b (127.16.79.65:39209): tablet 0a9c2cb33ad345678f1ffb71fcbd7303 (table test-table [id=8bb38542fbc44dc285a28c8ded57f1fc]) successfully deleted
I20260619 06:44:27.895859 18993 ts_tablet_manager.cc:1918] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:44:27.896075 18993 log.cc:1196] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/0a9c2cb33ad345678f1ffb71fcbd7303
I20260619 06:44:27.896656 18993 ts_tablet_manager.cc:1939] T 0a9c2cb33ad345678f1ffb71fcbd7303 P c05381a1ce2f43c0bdff51d620b89c2d: Deleting consensus metadata
I20260619 06:44:27.898417 18425 catalog_manager.cc:4841] TS c05381a1ce2f43c0bdff51d620b89c2d (127.16.79.67:45515): tablet 0a9c2cb33ad345678f1ffb71fcbd7303 (table test-table [id=8bb38542fbc44dc285a28c8ded57f1fc]) successfully deleted
I20260619 06:44:28.042850 18991 raft_consensus.cc:482] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:44:28.043098 18991 raft_consensus.cc:504] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:28.044075 18991 leader_election.cc:288] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aaccd506de1c4a28b6d1b18b2cf7244b (127.16.79.65:39209), c05381a1ce2f43c0bdff51d620b89c2d (127.16.79.67:45515)
I20260619 06:44:28.044585 18598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241a22e17b8f49cc89d71b10fac1d7fa" candidate_uuid: "4ea302c03631485996ba1e7031de06be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" is_pre_election: true
I20260619 06:44:28.044909 18598 raft_consensus.cc:2454] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ea302c03631485996ba1e7031de06be in term 0.
I20260619 06:44:28.044878 18876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241a22e17b8f49cc89d71b10fac1d7fa" candidate_uuid: "4ea302c03631485996ba1e7031de06be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" is_pre_election: true
I20260619 06:44:28.045101 18876 raft_consensus.cc:2454] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ea302c03631485996ba1e7031de06be in term 0.
I20260619 06:44:28.045589 18668 leader_election.cc:302] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ea302c03631485996ba1e7031de06be, c05381a1ce2f43c0bdff51d620b89c2d; no voters:
I20260619 06:44:28.045881 18991 raft_consensus.cc:2781] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:44:28.046021 18991 raft_consensus.cc:482] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:44:28.046120 18991 raft_consensus.cc:3037] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:28.049762 18991 raft_consensus.cc:504] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:28.050673 18991 leader_election.cc:288] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [CANDIDATE]: Term 1 election: Requested vote from peers aaccd506de1c4a28b6d1b18b2cf7244b (127.16.79.65:39209), c05381a1ce2f43c0bdff51d620b89c2d (127.16.79.67:45515)
I20260619 06:44:28.051074 18876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241a22e17b8f49cc89d71b10fac1d7fa" candidate_uuid: "4ea302c03631485996ba1e7031de06be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c05381a1ce2f43c0bdff51d620b89c2d"
I20260619 06:44:28.051312 18876 raft_consensus.cc:3037] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:28.051313 18598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241a22e17b8f49cc89d71b10fac1d7fa" candidate_uuid: "4ea302c03631485996ba1e7031de06be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b"
I20260619 06:44:28.051545 18598 raft_consensus.cc:3037] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:28.054557 18876 raft_consensus.cc:2454] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ea302c03631485996ba1e7031de06be in term 1.
I20260619 06:44:28.054642 18598 raft_consensus.cc:2454] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ea302c03631485996ba1e7031de06be in term 1.
I20260619 06:44:28.055013 18668 leader_election.cc:302] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ea302c03631485996ba1e7031de06be, c05381a1ce2f43c0bdff51d620b89c2d; no voters:
I20260619 06:44:28.055394 18991 raft_consensus.cc:2781] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:28.055558 18991 raft_consensus.cc:686] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [term 1 LEADER]: Becoming Leader. State: Replica: 4ea302c03631485996ba1e7031de06be, State: Running, Role: LEADER
I20260619 06:44:28.055903 18991 consensus_queue.cc:237] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:28.059324 18439 catalog_manager.cc:5495] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ea302c03631485996ba1e7031de06be (127.16.79.66). New cstate: current_term: 1 leader_uuid: "4ea302c03631485996ba1e7031de06be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } health_report { overall_health: UNKNOWN } } }
I20260619 06:44:28.109061 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:28.102437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54252 (local address 127.16.79.66:41289)
0619 06:44:28.102618 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:28.102625 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:28.102645 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:44:28.102713 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:28.102719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:28.102821 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:28.102926 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:44:28.103288 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:28.104641 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:28.106577 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:28.107385 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:28.107522 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:28.108116 (+ 594us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:44:28.108329 (+ 213us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:44:28.108428 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:28.108612 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 06:44:28.108704 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 06:44:28.124320 18989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:28.118384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:41289 (local address 127.16.79.66:50025)
0619 06:44:28.118561 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:28.118579 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:28.118690 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:28.119094 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:28.119100 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:28.119115 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:44:28.119316 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:28.119330 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:28.120777 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:28.120780 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:28.121723 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:28.121733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:28.122791 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:28.122798 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:28.122982 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:28.123039 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:44:28.123627 (+ 588us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:44:28.123640 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 06:44:28.123799 (+ 159us) client_negotiation.cc:240] Negotiation successful
0619 06:44:28.123941 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 06:44:28.124864 18944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:28.118610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:50025 (local address 127.16.79.66:41289)
0619 06:44:28.118784 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:28.118790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:28.118811 (+ 21us) server_negotiation.cc:407] Connection header received
0619 06:44:28.118870 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:28.118875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:28.118959 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:28.119067 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:44:28.119475 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:28.120644 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:28.121880 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:28.122591 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:28.123197 (+ 606us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:28.123305 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:44:28.123456 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:44:28.123592 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:28.123848 (+ 256us) server_negotiation.cc:299] Negotiation successful
0619 06:44:28.124007 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:44:28.135627 18598 raft_consensus.cc:1261] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 FOLLOWER]: Refusing update from remote peer 4ea302c03631485996ba1e7031de06be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:28.136332 18876 raft_consensus.cc:1261] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Refusing update from remote peer 4ea302c03631485996ba1e7031de06be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:28.136245 18991 consensus_queue.cc:1035] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:28.137141 18991 consensus_queue.cc:1035] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [LEADER]: Connected to new peer: Peer: permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:28.143510 18975 mvcc.cc:205] Tried to move back new op lower bound from 7298463613478453248 to 7298463613160439808. Current Snapshot: MvccSnapshot[applied={T|T < 7298463613478453248}]
W20260619 06:44:28.162266 18974 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 06:44:29.767412 18439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:44:29.767688 18439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:44:29.771546 18439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a: Sending DeleteTablet for 3 replicas of tablet 241a22e17b8f49cc89d71b10fac1d7fa
I20260619 06:44:29.772388 18578 tablet_service.cc:1514] Processing DeleteTablet for tablet 241a22e17b8f49cc89d71b10fac1d7fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:29 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:38412
I20260619 06:44:29.772440 18717 tablet_service.cc:1514] Processing DeleteTablet for tablet 241a22e17b8f49cc89d71b10fac1d7fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:29 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:54236
I20260619 06:44:29.772441 18856 tablet_service.cc:1514] Processing DeleteTablet for tablet 241a22e17b8f49cc89d71b10fac1d7fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:29 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:46640
I20260619 06:44:29.783550 19026 tablet_replica.cc:330] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d: stopping tablet replica
I20260619 06:44:29.783888 19026 raft_consensus.cc:2229] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:44:29.783927 19025 tablet_replica.cc:330] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be: stopping tablet replica
I20260619 06:44:29.784189 19025 raft_consensus.cc:2229] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:44:29.784222 19026 raft_consensus.cc:2258] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:29.784830 19025 raft_consensus.cc:2258] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:29.785271 19026 ts_tablet_manager.cc:1905] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:29.785435 19024 tablet_replica.cc:330] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b: stopping tablet replica
I20260619 06:44:29.785661 19024 raft_consensus.cc:2229] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:44:29.785903 19024 raft_consensus.cc:2258] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:29.786942 19024 ts_tablet_manager.cc:1905] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:29.788743 19025 ts_tablet_manager.cc:1905] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:29.791205 19026 ts_tablet_manager.cc:1918] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 06:44:29.791335 19026 log.cc:1196] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/241a22e17b8f49cc89d71b10fac1d7fa
I20260619 06:44:29.791747 19026 ts_tablet_manager.cc:1939] T 241a22e17b8f49cc89d71b10fac1d7fa P c05381a1ce2f43c0bdff51d620b89c2d: Deleting consensus metadata
I20260619 06:44:29.792958 18425 catalog_manager.cc:4841] TS c05381a1ce2f43c0bdff51d620b89c2d (127.16.79.67:45515): tablet 241a22e17b8f49cc89d71b10fac1d7fa (table test-table [id=21a598ab744e4b8a8b388ef61be87621]) successfully deleted
I20260619 06:44:29.794199 19024 ts_tablet_manager.cc:1918] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 06:44:29.794339 19024 log.cc:1196] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/241a22e17b8f49cc89d71b10fac1d7fa
I20260619 06:44:29.794785 19024 ts_tablet_manager.cc:1939] T 241a22e17b8f49cc89d71b10fac1d7fa P aaccd506de1c4a28b6d1b18b2cf7244b: Deleting consensus metadata
I20260619 06:44:29.794914 19025 ts_tablet_manager.cc:1918] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 06:44:29.795048 19025 log.cc:1196] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/241a22e17b8f49cc89d71b10fac1d7fa
I20260619 06:44:29.795509 19025 ts_tablet_manager.cc:1939] T 241a22e17b8f49cc89d71b10fac1d7fa P 4ea302c03631485996ba1e7031de06be: Deleting consensus metadata
I20260619 06:44:29.795883 18428 catalog_manager.cc:4841] TS aaccd506de1c4a28b6d1b18b2cf7244b (127.16.79.65:39209): tablet 241a22e17b8f49cc89d71b10fac1d7fa (table test-table [id=21a598ab744e4b8a8b388ef61be87621]) successfully deleted
I20260619 06:44:29.796629 18428 catalog_manager.cc:4841] TS 4ea302c03631485996ba1e7031de06be (127.16.79.66:41289): tablet 241a22e17b8f49cc89d71b10fac1d7fa (table test-table [id=21a598ab744e4b8a8b388ef61be87621]) successfully deleted
Jun 19 06:44:29 dist-test-slave-5w2x krb5kdc[18396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851469, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 06:44:29 dist-test-slave-5w2x krb5kdc[18396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851469, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:29.828500 19034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:29.814497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55742 (local address 127.16.79.126:43827)
0619 06:44:29.815477 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:29.815481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:29.815500 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:44:29.815549 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:29.815552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:29.815608 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:29.815691 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:29.817006 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:29.817744 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:29.818955 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:29.819387 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:29.819476 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:29.823193 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:29.823198 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:29.823201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:29.823233 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:29.825860 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:29.826540 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:29.826544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:29.826547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:29.826660 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:29.827001 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:29.827004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:29.827006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:29.827332 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:29.827599 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:29.827913 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 06:44:29.828121 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":879,"thread_start_us":720,"threads_started":1}
I20260619 06:44:29.831804 18439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:44:29.832397 18439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:44:29.838526 18578 tablet_service.cc:1467] Processing CreateTablet for tablet 6953d967a3c3445ca21a630d695d7df6 (DEFAULT_TABLE table=test-table [id=093c8ce65ef141049324d4d090e6db76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:29.838686 18856 tablet_service.cc:1467] Processing CreateTablet for tablet 6953d967a3c3445ca21a630d695d7df6 (DEFAULT_TABLE table=test-table [id=093c8ce65ef141049324d4d090e6db76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:29.838749 18717 tablet_service.cc:1467] Processing CreateTablet for tablet 6953d967a3c3445ca21a630d695d7df6 (DEFAULT_TABLE table=test-table [id=093c8ce65ef141049324d4d090e6db76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:29.838961 18578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6953d967a3c3445ca21a630d695d7df6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:29.839061 18856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6953d967a3c3445ca21a630d695d7df6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:29.839162 18717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6953d967a3c3445ca21a630d695d7df6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:29.845275 19035 tablet_bootstrap.cc:492] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be: Bootstrap starting.
I20260619 06:44:29.846532 19035 tablet_bootstrap.cc:654] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:29.847180 19037 tablet_bootstrap.cc:492] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d: Bootstrap starting.
I20260619 06:44:29.849339 19037 tablet_bootstrap.cc:654] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:29.850975 19036 tablet_bootstrap.cc:492] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b: Bootstrap starting.
I20260619 06:44:29.853021 19036 tablet_bootstrap.cc:654] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:29.853266 19037 tablet_bootstrap.cc:492] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d: No bootstrap required, opened a new log
I20260619 06:44:29.853441 19037 ts_tablet_manager.cc:1397] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260619 06:44:29.853897 19035 tablet_bootstrap.cc:492] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be: No bootstrap required, opened a new log
I20260619 06:44:29.854077 19035 ts_tablet_manager.cc:1397] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260619 06:44:29.854140 19037 raft_consensus.cc:348] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:29.854401 19037 raft_consensus.cc:374] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:29.854499 19037 raft_consensus.cc:729] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c05381a1ce2f43c0bdff51d620b89c2d, State: Initialized, Role: FOLLOWER
I20260619 06:44:29.854748 19037 consensus_queue.cc:260] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:29.854861 19035 raft_consensus.cc:348] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:29.855088 19035 raft_consensus.cc:374] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:29.855189 19035 raft_consensus.cc:729] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ea302c03631485996ba1e7031de06be, State: Initialized, Role: FOLLOWER
I20260619 06:44:29.855271 19036 tablet_bootstrap.cc:492] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b: No bootstrap required, opened a new log
I20260619 06:44:29.855401 19036 ts_tablet_manager.cc:1397] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 06:44:29.855468 19035 consensus_queue.cc:260] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:29.855720 19037 ts_tablet_manager.cc:1428] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 06:44:29.856173 19036 raft_consensus.cc:348] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:29.856421 19036 raft_consensus.cc:374] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:29.856511 19036 raft_consensus.cc:729] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaccd506de1c4a28b6d1b18b2cf7244b, State: Initialized, Role: FOLLOWER
I20260619 06:44:29.856623 19035 ts_tablet_manager.cc:1428] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260619 06:44:29.856762 19036 consensus_queue.cc:260] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:29.857606 19036 ts_tablet_manager.cc:1428] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 06:44:30.063637 19042 raft_consensus.cc:482] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:44:30.063897 19042 raft_consensus.cc:504] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:30.064718 19042 leader_election.cc:288] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ea302c03631485996ba1e7031de06be (127.16.79.66:41289), c05381a1ce2f43c0bdff51d620b89c2d (127.16.79.67:45515)
I20260619 06:44:30.065274 18738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6953d967a3c3445ca21a630d695d7df6" candidate_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ea302c03631485996ba1e7031de06be" is_pre_election: true
I20260619 06:44:30.065353 18874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6953d967a3c3445ca21a630d695d7df6" candidate_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" is_pre_election: true
I20260619 06:44:30.065515 18738 raft_consensus.cc:2454] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaccd506de1c4a28b6d1b18b2cf7244b in term 0.
I20260619 06:44:30.065639 18874 raft_consensus.cc:2454] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaccd506de1c4a28b6d1b18b2cf7244b in term 0.
I20260619 06:44:30.066077 18531 leader_election.cc:302] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ea302c03631485996ba1e7031de06be, aaccd506de1c4a28b6d1b18b2cf7244b; no voters:
I20260619 06:44:30.066406 19042 raft_consensus.cc:2781] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:44:30.066524 19042 raft_consensus.cc:482] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:44:30.066618 19042 raft_consensus.cc:3037] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:30.068818 19042 raft_consensus.cc:504] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:30.069455 19042 leader_election.cc:288] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [CANDIDATE]: Term 1 election: Requested vote from peers 4ea302c03631485996ba1e7031de06be (127.16.79.66:41289), c05381a1ce2f43c0bdff51d620b89c2d (127.16.79.67:45515)
I20260619 06:44:30.070012 18738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6953d967a3c3445ca21a630d695d7df6" candidate_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ea302c03631485996ba1e7031de06be"
I20260619 06:44:30.070076 18874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6953d967a3c3445ca21a630d695d7df6" candidate_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c05381a1ce2f43c0bdff51d620b89c2d"
I20260619 06:44:30.070238 18738 raft_consensus.cc:3037] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:30.070274 18874 raft_consensus.cc:3037] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:30.072355 18738 raft_consensus.cc:2454] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaccd506de1c4a28b6d1b18b2cf7244b in term 1.
I20260619 06:44:30.072361 18874 raft_consensus.cc:2454] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaccd506de1c4a28b6d1b18b2cf7244b in term 1.
I20260619 06:44:30.072866 18528 leader_election.cc:302] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4ea302c03631485996ba1e7031de06be, aaccd506de1c4a28b6d1b18b2cf7244b, c05381a1ce2f43c0bdff51d620b89c2d; no voters:
I20260619 06:44:30.073212 19042 raft_consensus.cc:2781] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:30.073647 19042 raft_consensus.cc:686] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 LEADER]: Becoming Leader. State: Replica: aaccd506de1c4a28b6d1b18b2cf7244b, State: Running, Role: LEADER
I20260619 06:44:30.073951 19042 consensus_queue.cc:237] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:30.076642 18439 catalog_manager.cc:5495] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b reported cstate change: term changed from 0 to 1, leader changed from <none> to aaccd506de1c4a28b6d1b18b2cf7244b (127.16.79.65). New cstate: current_term: 1 leader_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } health_report { overall_health: UNKNOWN } } }
I20260619 06:44:30.151038 19046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:30.146119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38420 (local address 127.16.79.65:39209)
0619 06:44:30.146753 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:30.146757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:30.146783 (+ 26us) server_negotiation.cc:407] Connection header received
0619 06:44:30.146834 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:30.146837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:30.146899 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:30.146989 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:44:30.147294 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:30.148072 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:30.149394 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:30.149825 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:30.149922 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:30.150307 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:44:30.150409 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:44:30.150483 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:30.150599 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 06:44:30.150657 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":341,"threads_started":1}
I20260619 06:44:30.159415 18874 raft_consensus.cc:1261] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Refusing update from remote peer aaccd506de1c4a28b6d1b18b2cf7244b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:30.159631 18738 raft_consensus.cc:1261] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Refusing update from remote peer aaccd506de1c4a28b6d1b18b2cf7244b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:30.159973 19043 consensus_queue.cc:1035] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:30.160398 19042 consensus_queue.cc:1035] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:30.203438 18439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:44:30.203697 18439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:44:30.206450 18439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a: Sending DeleteTablet for 3 replicas of tablet 6953d967a3c3445ca21a630d695d7df6
I20260619 06:44:30.207365 18856 tablet_service.cc:1514] Processing DeleteTablet for tablet 6953d967a3c3445ca21a630d695d7df6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:30 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:46640
I20260619 06:44:30.207366 18578 tablet_service.cc:1514] Processing DeleteTablet for tablet 6953d967a3c3445ca21a630d695d7df6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:30 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:38412
I20260619 06:44:30.207620 19026 tablet_replica.cc:330] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d: stopping tablet replica
I20260619 06:44:30.207625 19024 tablet_replica.cc:330] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b: stopping tablet replica
I20260619 06:44:30.207584 18717 tablet_service.cc:1514] Processing DeleteTablet for tablet 6953d967a3c3445ca21a630d695d7df6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:30 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:54236
I20260619 06:44:30.207798 19026 raft_consensus.cc:2229] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:44:30.207803 19024 raft_consensus.cc:2229] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:44:30.207885 19025 tablet_replica.cc:330] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be: stopping tablet replica
I20260619 06:44:30.207985 19026 raft_consensus.cc:2258] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:30.208065 19025 raft_consensus.cc:2229] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:44:30.208107 19024 raft_consensus.cc:2258] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:30.208266 19025 raft_consensus.cc:2258] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:30.209268 19025 ts_tablet_manager.cc:1905] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:30.209163 18439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:44:30.209970 18439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:44:30.215819 19025 ts_tablet_manager.cc:1918] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:44:30.215945 19025 log.cc:1196] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/6953d967a3c3445ca21a630d695d7df6
I20260619 06:44:30.216332 19025 ts_tablet_manager.cc:1939] T 6953d967a3c3445ca21a630d695d7df6 P 4ea302c03631485996ba1e7031de06be: Deleting consensus metadata
I20260619 06:44:30.217325 18428 catalog_manager.cc:4841] TS 4ea302c03631485996ba1e7031de06be (127.16.79.66:41289): tablet 6953d967a3c3445ca21a630d695d7df6 (table test-table [id=093c8ce65ef141049324d4d090e6db76]) successfully deleted
I20260619 06:44:30.217991 19024 ts_tablet_manager.cc:1905] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:30.220674 19026 ts_tablet_manager.cc:1905] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:30.225610 18717 tablet_service.cc:1467] Processing CreateTablet for tablet 463f22c20e164eeba79514c6eeb3fe8f (DEFAULT_TABLE table=test-table [id=001e8b4312aa4d7387a9e1ff4a399956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:30.226078 18717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 463f22c20e164eeba79514c6eeb3fe8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:30.226974 18856 tablet_service.cc:1467] Processing CreateTablet for tablet 463f22c20e164eeba79514c6eeb3fe8f (DEFAULT_TABLE table=test-table [id=001e8b4312aa4d7387a9e1ff4a399956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:30.227399 18856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 463f22c20e164eeba79514c6eeb3fe8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:30.225309 18578 tablet_service.cc:1467] Processing CreateTablet for tablet 463f22c20e164eeba79514c6eeb3fe8f (DEFAULT_TABLE table=test-table [id=001e8b4312aa4d7387a9e1ff4a399956]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:30.233142 18578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 463f22c20e164eeba79514c6eeb3fe8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:30.234088 19037 tablet_bootstrap.cc:492] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d: Bootstrap starting.
I20260619 06:44:30.235169 19026 ts_tablet_manager.cc:1918] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:44:30.235333 19026 log.cc:1196] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/6953d967a3c3445ca21a630d695d7df6
I20260619 06:44:30.235764 19026 ts_tablet_manager.cc:1939] T 6953d967a3c3445ca21a630d695d7df6 P c05381a1ce2f43c0bdff51d620b89c2d: Deleting consensus metadata
I20260619 06:44:30.235883 19037 tablet_bootstrap.cc:654] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:30.236828 18425 catalog_manager.cc:4841] TS c05381a1ce2f43c0bdff51d620b89c2d (127.16.79.67:45515): tablet 6953d967a3c3445ca21a630d695d7df6 (table test-table [id=093c8ce65ef141049324d4d090e6db76]) successfully deleted
I20260619 06:44:30.224704 19024 ts_tablet_manager.cc:1918] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:44:30.240429 19024 log.cc:1196] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/6953d967a3c3445ca21a630d695d7df6
I20260619 06:44:30.240602 19036 tablet_bootstrap.cc:492] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b: Bootstrap starting.
I20260619 06:44:30.240851 19024 ts_tablet_manager.cc:1939] T 6953d967a3c3445ca21a630d695d7df6 P aaccd506de1c4a28b6d1b18b2cf7244b: Deleting consensus metadata
I20260619 06:44:30.242357 19035 tablet_bootstrap.cc:492] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be: Bootstrap starting.
I20260619 06:44:30.242719 19036 tablet_bootstrap.cc:654] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:30.243757 19037 tablet_bootstrap.cc:492] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d: No bootstrap required, opened a new log
I20260619 06:44:30.243742 18428 catalog_manager.cc:4841] TS aaccd506de1c4a28b6d1b18b2cf7244b (127.16.79.65:39209): tablet 6953d967a3c3445ca21a630d695d7df6 (table test-table [id=093c8ce65ef141049324d4d090e6db76]) successfully deleted
I20260619 06:44:30.243965 19037 ts_tablet_manager.cc:1397] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.009s
I20260619 06:44:30.244380 19035 tablet_bootstrap.cc:654] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:30.244786 19037 raft_consensus.cc:348] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:30.245048 19037 raft_consensus.cc:374] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:30.245162 19037 raft_consensus.cc:729] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c05381a1ce2f43c0bdff51d620b89c2d, State: Initialized, Role: FOLLOWER
I20260619 06:44:30.245440 19037 consensus_queue.cc:260] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:30.245976 19037 ts_tablet_manager.cc:1428] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 06:44:30.247648 19036 tablet_bootstrap.cc:492] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b: No bootstrap required, opened a new log
I20260619 06:44:30.247823 19036 ts_tablet_manager.cc:1397] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.001s
I20260619 06:44:30.248567 19036 raft_consensus.cc:348] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:30.248843 19036 raft_consensus.cc:374] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:30.248935 19036 raft_consensus.cc:729] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaccd506de1c4a28b6d1b18b2cf7244b, State: Initialized, Role: FOLLOWER
I20260619 06:44:30.249172 19036 consensus_queue.cc:260] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:30.250243 19036 ts_tablet_manager.cc:1428] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 06:44:30.251494 19035 tablet_bootstrap.cc:492] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be: No bootstrap required, opened a new log
I20260619 06:44:30.251688 19035 ts_tablet_manager.cc:1397] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20260619 06:44:30.252365 19035 raft_consensus.cc:348] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:30.252581 19035 raft_consensus.cc:374] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:30.252678 19035 raft_consensus.cc:729] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ea302c03631485996ba1e7031de06be, State: Initialized, Role: FOLLOWER
I20260619 06:44:30.252962 19035 consensus_queue.cc:260] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:30.253484 19035 ts_tablet_manager.cc:1428] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 06:44:30.364920 19050 raft_consensus.cc:482] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:44:30.365120 19050 raft_consensus.cc:504] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:30.365871 19050 leader_election.cc:288] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ea302c03631485996ba1e7031de06be (127.16.79.66:41289), c05381a1ce2f43c0bdff51d620b89c2d (127.16.79.67:45515)
I20260619 06:44:30.366556 18738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "463f22c20e164eeba79514c6eeb3fe8f" candidate_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ea302c03631485996ba1e7031de06be" is_pre_election: true
I20260619 06:44:30.366578 18874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "463f22c20e164eeba79514c6eeb3fe8f" candidate_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" is_pre_election: true
I20260619 06:44:30.366793 18738 raft_consensus.cc:2454] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaccd506de1c4a28b6d1b18b2cf7244b in term 0.
I20260619 06:44:30.366839 18874 raft_consensus.cc:2454] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaccd506de1c4a28b6d1b18b2cf7244b in term 0.
I20260619 06:44:30.367233 18531 leader_election.cc:302] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ea302c03631485996ba1e7031de06be, aaccd506de1c4a28b6d1b18b2cf7244b; no voters:
I20260619 06:44:30.367482 19050 raft_consensus.cc:2781] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:44:30.367605 19050 raft_consensus.cc:482] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:44:30.367722 19050 raft_consensus.cc:3037] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:30.369791 19050 raft_consensus.cc:504] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:30.370424 19050 leader_election.cc:288] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [CANDIDATE]: Term 1 election: Requested vote from peers 4ea302c03631485996ba1e7031de06be (127.16.79.66:41289), c05381a1ce2f43c0bdff51d620b89c2d (127.16.79.67:45515)
I20260619 06:44:30.370920 18738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "463f22c20e164eeba79514c6eeb3fe8f" candidate_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ea302c03631485996ba1e7031de06be"
I20260619 06:44:30.371037 18874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "463f22c20e164eeba79514c6eeb3fe8f" candidate_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c05381a1ce2f43c0bdff51d620b89c2d"
I20260619 06:44:30.371129 18738 raft_consensus.cc:3037] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:30.371256 18874 raft_consensus.cc:3037] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:30.373466 18738 raft_consensus.cc:2454] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaccd506de1c4a28b6d1b18b2cf7244b in term 1.
I20260619 06:44:30.373522 18874 raft_consensus.cc:2454] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaccd506de1c4a28b6d1b18b2cf7244b in term 1.
I20260619 06:44:30.373920 18531 leader_election.cc:302] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ea302c03631485996ba1e7031de06be, aaccd506de1c4a28b6d1b18b2cf7244b; no voters:
I20260619 06:44:30.374228 19050 raft_consensus.cc:2781] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:30.374372 19050 raft_consensus.cc:686] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 LEADER]: Becoming Leader. State: Replica: aaccd506de1c4a28b6d1b18b2cf7244b, State: Running, Role: LEADER
I20260619 06:44:30.374624 19050 consensus_queue.cc:237] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } }
I20260619 06:44:30.377414 18441 catalog_manager.cc:5495] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b reported cstate change: term changed from 0 to 1, leader changed from <none> to aaccd506de1c4a28b6d1b18b2cf7244b (127.16.79.65). New cstate: current_term: 1 leader_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aaccd506de1c4a28b6d1b18b2cf7244b" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 } health_report { overall_health: UNKNOWN } } }
I20260619 06:44:30.421739 18874 raft_consensus.cc:1261] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Refusing update from remote peer aaccd506de1c4a28b6d1b18b2cf7244b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:30.422055 18738 raft_consensus.cc:1261] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Refusing update from remote peer aaccd506de1c4a28b6d1b18b2cf7244b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:30.422302 19050 consensus_queue.cc:1035] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c05381a1ce2f43c0bdff51d620b89c2d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:30.422578 19043 consensus_queue.cc:1035] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ea302c03631485996ba1e7031de06be" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 06:44:30.441275 19009 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 06:44:32.020052 18441 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:44:32.020287 18441 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:44:32.023839 18441 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93df491966b64691addd7608676ab37a: Sending DeleteTablet for 3 replicas of tablet 463f22c20e164eeba79514c6eeb3fe8f
I20260619 06:44:32.024587 18578 tablet_service.cc:1514] Processing DeleteTablet for tablet 463f22c20e164eeba79514c6eeb3fe8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:32 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:38412
I20260619 06:44:32.024721 18717 tablet_service.cc:1514] Processing DeleteTablet for tablet 463f22c20e164eeba79514c6eeb3fe8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:32 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:54236
I20260619 06:44:32.024858 18856 tablet_service.cc:1514] Processing DeleteTablet for tablet 463f22c20e164eeba79514c6eeb3fe8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:32 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:46640
I20260619 06:44:32.025565 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 18512
I20260619 06:44:32.037753 19078 tablet_replica.cc:330] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b: stopping tablet replica
I20260619 06:44:32.038359 19080 tablet_replica.cc:330] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d: stopping tablet replica
I20260619 06:44:32.038354 19078 raft_consensus.cc:2229] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:44:32.038697 19080 raft_consensus.cc:2229] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:44:32.038833 19078 raft_consensus.cc:2258] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:32.039005 19080 raft_consensus.cc:2258] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:32.039551 19079 tablet_replica.cc:330] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be: stopping tablet replica
I20260619 06:44:32.039733 19079 raft_consensus.cc:2229] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:44:32.039953 19078 ts_tablet_manager.cc:1905] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:32.039991 19079 raft_consensus.cc:2258] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:32.040751 19080 ts_tablet_manager.cc:1905] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:32.042985 19079 ts_tablet_manager.cc:1905] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:32.046749 19078 ts_tablet_manager.cc:1918] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 06:44:32.046892 19078 log.cc:1196] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/463f22c20e164eeba79514c6eeb3fe8f
I20260619 06:44:32.047237 19080 ts_tablet_manager.cc:1918] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 06:44:32.047353 19078 ts_tablet_manager.cc:1939] T 463f22c20e164eeba79514c6eeb3fe8f P aaccd506de1c4a28b6d1b18b2cf7244b: Deleting consensus metadata
I20260619 06:44:32.047386 19080 log.cc:1196] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/463f22c20e164eeba79514c6eeb3fe8f
I20260619 06:44:32.047762 19080 ts_tablet_manager.cc:1939] T 463f22c20e164eeba79514c6eeb3fe8f P c05381a1ce2f43c0bdff51d620b89c2d: Deleting consensus metadata
I20260619 06:44:32.048520 18428 catalog_manager.cc:4841] TS aaccd506de1c4a28b6d1b18b2cf7244b (127.16.79.65:39209): tablet 463f22c20e164eeba79514c6eeb3fe8f (table test-table [id=001e8b4312aa4d7387a9e1ff4a399956]) successfully deleted
I20260619 06:44:32.048812 18425 catalog_manager.cc:4841] TS c05381a1ce2f43c0bdff51d620b89c2d (127.16.79.67:45515): tablet 463f22c20e164eeba79514c6eeb3fe8f (table test-table [id=001e8b4312aa4d7387a9e1ff4a399956]) successfully deleted
I20260619 06:44:32.048911 19079 ts_tablet_manager.cc:1918] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 06:44:32.049070 19079 log.cc:1196] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/463f22c20e164eeba79514c6eeb3fe8f
I20260619 06:44:32.049470 19079 ts_tablet_manager.cc:1939] T 463f22c20e164eeba79514c6eeb3fe8f P 4ea302c03631485996ba1e7031de06be: Deleting consensus metadata
I20260619 06:44:32.050462 18428 catalog_manager.cc:4841] TS 4ea302c03631485996ba1e7031de06be (127.16.79.66:41289): tablet 463f22c20e164eeba79514c6eeb3fe8f (table test-table [id=001e8b4312aa4d7387a9e1ff4a399956]) successfully deleted
Jun 19 06:44:32 dist-test-slave-5w2x krb5kdc[18396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851469, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:44:32.059139 19082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:32.028336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38422 (local address 127.16.79.65:39209)
0619 06:44:32.038637 (+ 10301us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:32.038645 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:32.038676 (+ 31us) server_negotiation.cc:407] Connection header received
0619 06:44:32.038753 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:32.038761 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:32.038890 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:32.039027 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:32.041404 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:32.042265 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:32.044157 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:32.044888 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:32.045015 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:32.053494 (+ 8479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:32.053511 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:32.053527 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:32.053562 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:32.056342 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:32.057050 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:32.057058 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:32.057061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:32.057228 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:32.057643 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:32.057647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:32.057649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:32.058011 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:32.058265 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:32.058551 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 06:44:32.058718 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10124,"thread_start_us":9135,"threads_started":1}
W20260619 06:44:32.059857 18643 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38422
W20260619 06:44:32.060515 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 18512: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:44:32.060745 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 18512
I20260619 06:44:32.083047 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 18652
Jun 19 06:44:32 dist-test-slave-5w2x krb5kdc[18396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851469, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:44:32.098109 19083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:32.083842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54254 (local address 127.16.79.66:41289)
0619 06:44:32.084371 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:32.084375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:32.084391 (+ 16us) server_negotiation.cc:407] Connection header received
0619 06:44:32.084443 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:32.084446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:32.084546 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:32.084684 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:32.086050 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:32.086920 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:32.088271 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:32.088670 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:32.088779 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:32.092787 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:32.092800 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:32.092832 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:32.092870 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:32.095643 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:32.096460 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:32.096467 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:32.096470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:32.096585 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:32.096907 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:32.096910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:32.096912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:32.097245 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:32.097455 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:32.097673 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 06:44:32.097850 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":268,"threads_started":1}
W20260619 06:44:32.098829 18782 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54254
W20260619 06:44:32.099457 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 18652: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:44:32.099581 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 18652
I20260619 06:44:32.123898 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 18791
Jun 19 06:44:32 dist-test-slave-5w2x krb5kdc[18396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851469, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:44:32.140275 19084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:32.124674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46642 (local address 127.16.79.67:45515)
0619 06:44:32.125190 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:32.125194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:32.125221 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:44:32.125275 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:32.125279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:32.125336 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:32.125427 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:32.126746 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:32.127566 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:32.128913 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:32.129419 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:32.129516 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:32.133674 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:32.133690 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:32.133707 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:32.133752 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:32.136637 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:32.137401 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:32.137409 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:32.137412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:32.137566 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:32.138090 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:32.138096 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:32.138102 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:32.138631 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:32.138952 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:32.139524 (+ 572us) server_negotiation.cc:299] Negotiation successful
0619 06:44:32.139757 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":272,"threads_started":1}
W20260619 06:44:32.141129 18921 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46642
W20260619 06:44:32.141824 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 18791: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:44:32.141999 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 18791
I20260619 06:44:32.158886 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 18409
I20260619 06:44:32.159993 18481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:32.273435 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 18409
2026-06-19T06:44:32Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8372 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:44:32 dist-test-slave-5w2x krb5kdc[19088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:44:32 dist-test-slave-5w2x krb5kdc[19088](info): set up 2 sockets
Jun 19 06:44:32 dist-test-slave-5w2x krb5kdc[19088](info): commencing operation
krb5kdc: starting...
W20260619 06:44:33.368402 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.003s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:44:33 dist-test-slave-5w2x krb5kdc[19088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:44:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:44:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:44:33.551647 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:42551
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:36597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:42551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:33.867091 19101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:33.867420 19101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:33.867587 19101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:33.874711 19101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:44:33.874851 19101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:33.874963 19101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:33.875063 19101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:44:33.875142 19101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:44:33.884804 19101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:36597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:42551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:42551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:33.886073 19101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:33.887346 19101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:33.898571 19106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:33.898542 19107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:33.898613 19109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:33.899382 19101 server_base.cc:1034] running on GCE node
I20260619 06:44:33.900239 19101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:33.901741 19101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:33.902982 19101 hybrid_clock.cc:648] HybridClock initialized: now 1781851473902905 us; error 70 us; skew 500 ppm
I20260619 06:44:33.903374 19101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:33 dist-test-slave-5w2x krb5kdc[19088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851473, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:33.906738 19101 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:44:33.908329 19101 webserver.cc:460] Webserver started at http://127.16.79.126:41473/ using document root <none> and password file <none>
I20260619 06:44:33.909022 19101 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:33.909164 19101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:33.909487 19101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:33.912427 19101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "92f2f440eb1141099d0ba803e21ccb22"
format_stamp: "Formatted at 2026-06-19 06:44:33 on dist-test-slave-5w2x"
server_key: "406757f5a76c8629c0c0ff2fb0322b26"
server_key_iv: "ec1b17b5e41f1b9f798f4e67a0437fe6"
server_key_version: "encryptionkey@0"
I20260619 06:44:33.913160 19101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "92f2f440eb1141099d0ba803e21ccb22"
format_stamp: "Formatted at 2026-06-19 06:44:33 on dist-test-slave-5w2x"
server_key: "406757f5a76c8629c0c0ff2fb0322b26"
server_key_iv: "ec1b17b5e41f1b9f798f4e67a0437fe6"
server_key_version: "encryptionkey@0"
I20260619 06:44:33.918581 19101 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 06:44:33.922032 19116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:33.923404 19101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:44:33.923605 19101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "92f2f440eb1141099d0ba803e21ccb22"
format_stamp: "Formatted at 2026-06-19 06:44:33 on dist-test-slave-5w2x"
server_key: "406757f5a76c8629c0c0ff2fb0322b26"
server_key_iv: "ec1b17b5e41f1b9f798f4e67a0437fe6"
server_key_version: "encryptionkey@0"
I20260619 06:44:33.923805 19101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:33.956447 19101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:33.961066 19101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:33.961377 19101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:33.979895 19101 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:42551
I20260619 06:44:33.979902 19167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:42551 every 8 connection(s)
I20260619 06:44:33.981324 19101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:44:33.985024 19168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:33.988837 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 19101
I20260619 06:44:33.989082 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:44:33.989884 16701 external_mini_cluster.cc:1428] Setting key 6a4d7ddf8d46ac03eaead5059a18010c
I20260619 06:44:33.999095 19168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22: Bootstrap starting.
I20260619 06:44:34.003284 19168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:34.004777 19168 log.cc:824] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22: Log is configured to *not* fsync() on all Append() calls
Jun 19 06:44:34 dist-test-slave-5w2x krb5kdc[19088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:34.007826 19168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22: No bootstrap required, opened a new log
I20260619 06:44:34.013275 19171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:33.991881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34836 (local address 127.16.79.126:42551)
0619 06:44:33.992694 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:33.992706 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:33.992745 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:44:33.994317 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:33.994337 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:33.994746 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:33.995060 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:33.997099 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:33.998365 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:34.000284 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:34.000951 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:34.001086 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:34.006124 (+ 5038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:34.006132 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:34.006147 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:34.006180 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:34.009248 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:34.009917 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:34.009923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:34.009932 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:34.010100 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:34.010500 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:34.010509 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:34.010511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:34.011036 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:34.011353 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:34.011777 (+ 424us) server_negotiation.cc:299] Negotiation successful
0619 06:44:34.012262 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":415,"threads_started":1}
I20260619 06:44:34.016666 19168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92f2f440eb1141099d0ba803e21ccb22" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42551 } }
I20260619 06:44:34.017045 19168 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:34.017179 19168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f2f440eb1141099d0ba803e21ccb22, State: Initialized, Role: FOLLOWER
I20260619 06:44:34.017771 19168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92f2f440eb1141099d0ba803e21ccb22" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42551 } }
I20260619 06:44:34.018021 19168 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:44:34.018168 19168 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:44:34.018368 19168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:34.020859 19168 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92f2f440eb1141099d0ba803e21ccb22" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42551 } }
I20260619 06:44:34.021425 19168 leader_election.cc:302] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92f2f440eb1141099d0ba803e21ccb22; no voters:
I20260619 06:44:34.022022 19168 leader_election.cc:288] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:44:34.022121 19173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:34.023353 19173 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [term 1 LEADER]: Becoming Leader. State: Replica: 92f2f440eb1141099d0ba803e21ccb22, State: Running, Role: LEADER
I20260619 06:44:34.023902 19173 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92f2f440eb1141099d0ba803e21ccb22" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42551 } }
I20260619 06:44:34.024422 19168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:44:34.026613 19174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92f2f440eb1141099d0ba803e21ccb22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92f2f440eb1141099d0ba803e21ccb22" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42551 } } }
I20260619 06:44:34.026639 19175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92f2f440eb1141099d0ba803e21ccb22. Latest consensus state: current_term: 1 leader_uuid: "92f2f440eb1141099d0ba803e21ccb22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92f2f440eb1141099d0ba803e21ccb22" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42551 } } }
I20260619 06:44:34.027079 19175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:34.027079 19174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22 [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:34.029104 19178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:44:34.034412 19178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:44:34.043408 19178 catalog_manager.cc:1347] Generated new cluster ID: d30b1b86ac30485d815d2798917102bb
I20260619 06:44:34.043533 19178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:44:34.067173 19178 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:44:34.069133 19178 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:44:34.077083 19178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92f2f440eb1141099d0ba803e21ccb22: Generated new TSK 0
I20260619 06:44:34.078105 19178 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:44:34.125641 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42551
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:36597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:34.456228 19194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:34.456583 19194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:34.456795 19194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:34.464814 19194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:34.464959 19194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:34.465168 19194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:44:34.475199 19194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:36597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:34.476528 19194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:34.477881 19194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:34.490657 19199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:34.490747 19200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:34.490706 19202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:34.491684 19194 server_base.cc:1034] running on GCE node
I20260619 06:44:34.492388 19194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:34.493419 19194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:34.494611 19194 hybrid_clock.cc:648] HybridClock initialized: now 1781851474494565 us; error 38 us; skew 500 ppm
I20260619 06:44:34.494943 19194 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:34 dist-test-slave-5w2x krb5kdc[19088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851474, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:34.498526 19194 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:44:34.500056 19194 webserver.cc:460] Webserver started at http://127.16.79.65:33993/ using document root <none> and password file <none>
I20260619 06:44:34.500808 19194 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:34.500911 19194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:34.501251 19194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:34.504293 19194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "e9158271579047cb8e976dbfb7f30aad"
format_stamp: "Formatted at 2026-06-19 06:44:34 on dist-test-slave-5w2x"
server_key: "533d8bcaeb250a1ebd1f9497b9833879"
server_key_iv: "dfe486d98f4ec68e7b1245eb42f17f84"
server_key_version: "encryptionkey@0"
I20260619 06:44:34.505121 19194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "e9158271579047cb8e976dbfb7f30aad"
format_stamp: "Formatted at 2026-06-19 06:44:34 on dist-test-slave-5w2x"
server_key: "533d8bcaeb250a1ebd1f9497b9833879"
server_key_iv: "dfe486d98f4ec68e7b1245eb42f17f84"
server_key_version: "encryptionkey@0"
I20260619 06:44:34.510569 19194 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:44:34.514379 19209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:34.515894 19194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:44:34.516085 19194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "e9158271579047cb8e976dbfb7f30aad"
format_stamp: "Formatted at 2026-06-19 06:44:34 on dist-test-slave-5w2x"
server_key: "533d8bcaeb250a1ebd1f9497b9833879"
server_key_iv: "dfe486d98f4ec68e7b1245eb42f17f84"
server_key_version: "encryptionkey@0"
I20260619 06:44:34.516378 19194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:34.535472 19194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:34.539587 19194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:34.539884 19194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:34.540928 19194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:34.542274 19194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:34.542389 19194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:34.542536 19194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:34.542608 19194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:34.576309 19194 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:43071
I20260619 06:44:34.576337 19321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:43071 every 8 connection(s)
I20260619 06:44:34.577790 19194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:44:34.584525 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 19194
I20260619 06:44:34.584744 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:44:34.585389 16701 external_mini_cluster.cc:1428] Setting key 7917a1e0c10f20349735bebd93a91253
Jun 19 06:44:34 dist-test-slave-5w2x krb5kdc[19088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851474, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:34.600200 19324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:34.580332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:46665 (local address 127.16.79.126:42551)
0619 06:44:34.580734 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:34.580739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:34.581803 (+ 1064us) server_negotiation.cc:407] Connection header received
0619 06:44:34.583911 (+ 2108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:34.583917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:34.584018 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:34.584170 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:34.585911 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:34.586759 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:34.588395 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:34.588791 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:34.588940 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:34.593045 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:34.593055 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:34.593063 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:34.593109 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:34.596796 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:34.597622 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:34.597631 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:34.597637 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:34.597788 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:34.598216 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:34.598224 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:34.598230 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:34.598617 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:34.598768 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:34.599470 (+ 702us) server_negotiation.cc:299] Negotiation successful
0619 06:44:34.599743 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":211,"threads_started":1}
I20260619 06:44:34.601126 19325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:34.580966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:42551 (local address 127.16.79.65:46665)
0619 06:44:34.581642 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:34.581684 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:34.583540 (+ 1856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:34.584430 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:34.584438 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:34.585083 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:34.585728 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:34.585747 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:34.586935 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:34.586942 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:34.588250 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:34.588262 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:34.589040 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:34.589048 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:34.589291 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:34.589998 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:34.590026 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:34.592756 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:34.596990 (+ 4234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:34.596998 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:34.597012 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:34.597475 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:34.597897 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:34.597902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:34.597905 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:34.598086 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:34.598735 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:34.598741 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:34.599072 (+ 331us) client_negotiation.cc:769] Sending connection context
0619 06:44:34.599301 (+ 229us) client_negotiation.cc:240] Negotiation successful
0619 06:44:34.599830 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":263,"threads_started":1}
I20260619 06:44:34.602774 19322 heartbeater.cc:347] Connected to a master server at 127.16.79.126:42551
I20260619 06:44:34.603101 19322 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
I20260619 06:44:34.603919 19322 heartbeater.cc:511] Master 127.16.79.126:42551 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
I20260619 06:44:34.606427 19133 ts_manager.cc:194] Registered new tserver with Master: e9158271579047cb8e976dbfb7f30aad (127.16.79.65:43071)
I20260619 06:44:34.608711 19133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:46665
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:44:34.631517 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42551
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:36597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:34.948508 19328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:34.948781 19328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:34.948936 19328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:34.956068 19328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:34.956194 19328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:34.956377 19328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:44:34.966243 19328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:36597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:34.967502 19328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:34.968828 19328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:34.981419 19334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:34.981444 19336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:34.981551 19333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:34.985646 19328 server_base.cc:1034] running on GCE node
I20260619 06:44:34.986328 19328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:34.987298 19328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:34.988523 19328 hybrid_clock.cc:648] HybridClock initialized: now 1781851474988469 us; error 53 us; skew 500 ppm
I20260619 06:44:34.988838 19328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:34 dist-test-slave-5w2x krb5kdc[19088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851474, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:34.991987 19328 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:44:34.993543 19328 webserver.cc:460] Webserver started at http://127.16.79.66:35773/ using document root <none> and password file <none>
I20260619 06:44:34.994247 19328 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:34.994390 19328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:34.994689 19328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:34.997651 19328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "65a605e8c01e46e09a15d9f1becc7631"
format_stamp: "Formatted at 2026-06-19 06:44:34 on dist-test-slave-5w2x"
server_key: "b6f6f08f96db7d5039a2c0cc8226159d"
server_key_iv: "c77f7c52194f6dcf1f65b1b8a4c75b9a"
server_key_version: "encryptionkey@0"
I20260619 06:44:34.998404 19328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "65a605e8c01e46e09a15d9f1becc7631"
format_stamp: "Formatted at 2026-06-19 06:44:34 on dist-test-slave-5w2x"
server_key: "b6f6f08f96db7d5039a2c0cc8226159d"
server_key_iv: "c77f7c52194f6dcf1f65b1b8a4c75b9a"
server_key_version: "encryptionkey@0"
I20260619 06:44:35.003222 19328 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260619 06:44:35.006524 19343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:35.007783 19328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:44:35.007951 19328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "65a605e8c01e46e09a15d9f1becc7631"
format_stamp: "Formatted at 2026-06-19 06:44:34 on dist-test-slave-5w2x"
server_key: "b6f6f08f96db7d5039a2c0cc8226159d"
server_key_iv: "c77f7c52194f6dcf1f65b1b8a4c75b9a"
server_key_version: "encryptionkey@0"
I20260619 06:44:35.008179 19328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:35.034818 19328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:35.038745 19328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:35.039055 19328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:35.039989 19328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:35.041325 19328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:35.041427 19328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:35.041575 19328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:35.041647 19328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:35.073915 19328 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:38135
I20260619 06:44:35.073958 19455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:38135 every 8 connection(s)
I20260619 06:44:35.075171 19328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:44:35.079010 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 19328
I20260619 06:44:35.079205 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:44:35.079643 16701 external_mini_cluster.cc:1428] Setting key 9cdcdaa5bcf1577a1388eae6a80c3fb7
Jun 19 06:44:35 dist-test-slave-5w2x krb5kdc[19088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851474, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:35.094992 19324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:35.077560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:40481 (local address 127.16.79.126:42551)
0619 06:44:35.077761 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:35.077767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:35.078992 (+ 1225us) server_negotiation.cc:407] Connection header received
0619 06:44:35.081115 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:35.081120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:35.081209 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:35.081328 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:35.082738 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:35.083447 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:35.085059 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:35.085498 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:35.085633 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:35.089331 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:35.089336 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:35.089339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:35.089372 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:35.091928 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:35.092723 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:35.092732 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:35.092735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:35.092904 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:35.093337 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:35.093344 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:35.093351 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:35.093599 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:35.093748 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:35.094423 (+ 675us) server_negotiation.cc:299] Negotiation successful
0619 06:44:35.094648 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
I20260619 06:44:35.096020 19458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:35.078163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:42551 (local address 127.16.79.66:40481)
0619 06:44:35.078846 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:35.078883 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:35.080862 (+ 1979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:35.081544 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:35.081552 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:35.082038 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:35.082576 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:35.082593 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:35.083588 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:35.083595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:35.084911 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:35.084926 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:35.085711 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:35.085718 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:35.086008 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:35.086761 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:35.086782 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:35.089106 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:35.092089 (+ 2983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:35.092098 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:35.092113 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:35.092589 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:35.093020 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:35.093026 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:35.093030 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:35.093213 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:35.093721 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:35.093729 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:35.094067 (+ 338us) client_negotiation.cc:769] Sending connection context
0619 06:44:35.094330 (+ 263us) client_negotiation.cc:240] Negotiation successful
0619 06:44:35.094847 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":256,"threads_started":1}
I20260619 06:44:35.097538 19456 heartbeater.cc:347] Connected to a master server at 127.16.79.126:42551
I20260619 06:44:35.097838 19456 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
I20260619 06:44:35.098663 19456 heartbeater.cc:511] Master 127.16.79.126:42551 requested a full tablet report, sending...
I20260619 06:44:35.101719 19133 ts_manager.cc:194] Registered new tserver with Master: 65a605e8c01e46e09a15d9f1becc7631 (127.16.79.66:38135)
I20260619 06:44:35.102533 19133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:40481
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:44:35.121833 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42551
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:36597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:35.445436 19461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:35.445696 19461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:35.445794 19461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:35.452840 19461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:35.452939 19461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:35.453055 19461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:44:35.462502 19461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:36597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42551
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:35.463689 19461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:35.465072 19461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:35.477684 19466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:35.477619 19469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:35.477721 19467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:35.478787 19461 server_base.cc:1034] running on GCE node
I20260619 06:44:35.479382 19461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:35.480443 19461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:35.481673 19461 hybrid_clock.cc:648] HybridClock initialized: now 1781851475481593 us; error 75 us; skew 500 ppm
I20260619 06:44:35.482005 19461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:35 dist-test-slave-5w2x krb5kdc[19088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851475, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:35.485122 19461 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:44:35.486558 19461 webserver.cc:460] Webserver started at http://127.16.79.67:41601/ using document root <none> and password file <none>
I20260619 06:44:35.487268 19461 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:35.487412 19461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:35.487715 19461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:35.490556 19461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "f747c09f62b242058a86aeb7bb6c8eea"
format_stamp: "Formatted at 2026-06-19 06:44:35 on dist-test-slave-5w2x"
server_key: "fa7d8977f7e67fc33db11efa7fc754a5"
server_key_iv: "5f1959f6eb91a654d006e186b91900a4"
server_key_version: "encryptionkey@0"
I20260619 06:44:35.491313 19461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "f747c09f62b242058a86aeb7bb6c8eea"
format_stamp: "Formatted at 2026-06-19 06:44:35 on dist-test-slave-5w2x"
server_key: "fa7d8977f7e67fc33db11efa7fc754a5"
server_key_iv: "5f1959f6eb91a654d006e186b91900a4"
server_key_version: "encryptionkey@0"
I20260619 06:44:35.496443 19461 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260619 06:44:35.499709 19476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:35.501076 19461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:44:35.501245 19461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "f747c09f62b242058a86aeb7bb6c8eea"
format_stamp: "Formatted at 2026-06-19 06:44:35 on dist-test-slave-5w2x"
server_key: "fa7d8977f7e67fc33db11efa7fc754a5"
server_key_iv: "5f1959f6eb91a654d006e186b91900a4"
server_key_version: "encryptionkey@0"
I20260619 06:44:35.501435 19461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:35.520695 19461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:35.525106 19461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:35.525403 19461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:35.526378 19461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:35.527935 19461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:35.528045 19461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:35.528215 19461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:35.528301 19461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:35.561715 19461 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:36473
I20260619 06:44:35.561764 19588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:36473 every 8 connection(s)
I20260619 06:44:35.563014 19461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:44:35.570729 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 19461
I20260619 06:44:35.570935 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:44:35.571501 16701 external_mini_cluster.cc:1428] Setting key d057a35dddcc55e9179b34d055ed7e8f
Jun 19 06:44:35 dist-test-slave-5w2x krb5kdc[19088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851475, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:35.583086 19324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:35.565646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:51063 (local address 127.16.79.126:42551)
0619 06:44:35.565851 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:35.565857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:35.566988 (+ 1131us) server_negotiation.cc:407] Connection header received
0619 06:44:35.568802 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:35.568806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:35.568873 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:35.568961 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:35.570404 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:35.571597 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:35.573610 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:35.574053 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:35.574174 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:35.577811 (+ 3637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:35.577816 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:35.577819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:35.577873 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:35.580466 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:35.581286 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:35.581290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:35.581292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:35.581420 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:35.581773 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:35.581776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:35.581778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:35.582022 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:35.582127 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:35.582698 (+ 571us) server_negotiation.cc:299] Negotiation successful
0619 06:44:35.582847 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:44:35.584002 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:35.566166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:42551 (local address 127.16.79.67:51063)
0619 06:44:35.566823 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:35.566870 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:35.568523 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:35.569157 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:35.569164 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:35.569660 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:35.570198 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:35.570215 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:35.571737 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:35.571740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:35.573471 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:35.573487 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:35.574281 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:35.574289 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:35.574529 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:35.575164 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:35.575189 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:35.577607 (+ 2418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:35.580600 (+ 2993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:35.580610 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:35.580624 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:35.581175 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:35.581513 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:35.581516 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:35.581518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:35.581681 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:35.582123 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:35.582128 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:35.582408 (+ 280us) client_negotiation.cc:769] Sending connection context
0619 06:44:35.582615 (+ 207us) client_negotiation.cc:240] Negotiation successful
0619 06:44:35.582962 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":249,"threads_started":1}
I20260619 06:44:35.585582 19589 heartbeater.cc:347] Connected to a master server at 127.16.79.126:42551
I20260619 06:44:35.585878 19589 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:35.586694 19589 heartbeater.cc:511] Master 127.16.79.126:42551 requested a full tablet report, sending...
I20260619 06:44:35.588047 19133 ts_manager.cc:194] Registered new tserver with Master: f747c09f62b242058a86aeb7bb6c8eea (127.16.79.67:36473)
I20260619 06:44:35.588845 19133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:51063
I20260619 06:44:35.598917 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:44:35.612025 19322 heartbeater.cc:503] Master 127.16.79.126:42551 was elected leader, sending a full tablet report...
W20260619 06:44:35.628479 19324 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:44:35.625835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58044 (local address 127.16.79.126:42551)
0619 06:44:35.626027 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:35.626034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:35.626475 (+ 441us) server_negotiation.cc:407] Connection header received
0619 06:44:35.626688 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:35.626694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:35.626767 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:35.626899 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:35.628263 (+ 1364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58044: BlockingRecv error: recv got EOF from 127.0.0.1:58044 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:44:35.631796 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 19194
W20260619 06:44:35.634529 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 19194: Not authorized: Client connection negotiation failed: client connection to 127.16.79.65:43071: server requires authentication, but client does not have Kerberos credentials available
I20260619 06:44:35.634717 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 19194
I20260619 06:44:35.651790 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 19328
W20260619 06:44:35.654407 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 19328: Not authorized: Client connection negotiation failed: client connection to 127.16.79.66:38135: server requires authentication, but client does not have Kerberos credentials available
I20260619 06:44:35.654577 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 19328
W20260619 06:44:35.654764 19601 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:44:35.652706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60372 (local address 127.16.79.66:38135)
0619 06:44:35.653290 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:35.653296 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:35.653321 (+ 25us) server_negotiation.cc:407] Connection header received
0619 06:44:35.653383 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:35.653389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:35.653597 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:35.653721 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:35.654386 (+ 665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60372: BlockingRecv error: recv got EOF from 127.0.0.1:60372 (error 108)
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":307,"threads_started":1}
I20260619 06:44:35.669137 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 19461
W20260619 06:44:35.671911 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 19461: Not authorized: Client connection negotiation failed: client connection to 127.16.79.67:36473: server requires authentication, but client does not have Kerberos credentials available
I20260619 06:44:35.672048 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 19461
I20260619 06:44:35.687105 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 19101
I20260619 06:44:35.688336 19163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:35.789543 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 19101
2026-06-19T06:44:35Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3513 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:44:35 dist-test-slave-5w2x krb5kdc[19606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:44:35 dist-test-slave-5w2x krb5kdc[19606](info): set up 2 sockets
Jun 19 06:44:35 dist-test-slave-5w2x krb5kdc[19606](info): commencing operation
krb5kdc: starting...
W20260619 06:44:36.868435 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.003s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:44:36 dist-test-slave-5w2x krb5kdc[19606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:44:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:44:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:44:37.051724 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:32973
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:35993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:32973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:37.370074 19619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:37.370368 19619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:37.370556 19619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:37.377770 19619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:44:37.377884 19619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:37.378016 19619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:37.378118 19619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:44:37.378206 19619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:44:37.388221 19619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:35993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:32973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:32973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:37.389505 19619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:37.390836 19619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:37.402254 19625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:37.402279 19624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:37.402814 19627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:37.403040 19619 server_base.cc:1034] running on GCE node
I20260619 06:44:37.403851 19619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:37.405270 19619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:37.406507 19619 hybrid_clock.cc:648] HybridClock initialized: now 1781851477406471 us; error 57 us; skew 500 ppm
I20260619 06:44:37.406862 19619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:37 dist-test-slave-5w2x krb5kdc[19606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851477, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:37.414872 19619 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:44:37.416445 19619 webserver.cc:460] Webserver started at http://127.16.79.126:39971/ using document root <none> and password file <none>
I20260619 06:44:37.417152 19619 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:37.417310 19619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:37.417627 19619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:37.420657 19619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "c35d88b6b43f4f7f9c8154b5b6c26003"
format_stamp: "Formatted at 2026-06-19 06:44:37 on dist-test-slave-5w2x"
server_key: "089df47c07e1f3bf8c75f322bf76ae03"
server_key_iv: "d22f7dcbb5591ca98923a5fb04760329"
server_key_version: "encryptionkey@0"
I20260619 06:44:37.421443 19619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "c35d88b6b43f4f7f9c8154b5b6c26003"
format_stamp: "Formatted at 2026-06-19 06:44:37 on dist-test-slave-5w2x"
server_key: "089df47c07e1f3bf8c75f322bf76ae03"
server_key_iv: "d22f7dcbb5591ca98923a5fb04760329"
server_key_version: "encryptionkey@0"
I20260619 06:44:37.426836 19619 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:44:37.430475 19634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:37.431881 19619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:44:37.432089 19619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "c35d88b6b43f4f7f9c8154b5b6c26003"
format_stamp: "Formatted at 2026-06-19 06:44:37 on dist-test-slave-5w2x"
server_key: "089df47c07e1f3bf8c75f322bf76ae03"
server_key_iv: "d22f7dcbb5591ca98923a5fb04760329"
server_key_version: "encryptionkey@0"
I20260619 06:44:37.432377 19619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:37.449580 19619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:37.453583 19619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:37.453866 19619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:37.471727 19619 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:32973
I20260619 06:44:37.471763 19685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:32973 every 8 connection(s)
I20260619 06:44:37.473066 19619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:44:37.476711 19686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:37.478890 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 19619
I20260619 06:44:37.479115 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:44:37.479763 16701 external_mini_cluster.cc:1428] Setting key 22b7de562dcbd995a65fd908955c8429
I20260619 06:44:37.488359 19686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003: Bootstrap starting.
I20260619 06:44:37.492036 19686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:37.493371 19686 log.cc:824] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003: Log is configured to *not* fsync() on all Append() calls
Jun 19 06:44:37 dist-test-slave-5w2x krb5kdc[19606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:37.496079 19686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003: No bootstrap required, opened a new log
I20260619 06:44:37.502163 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:37.481706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40340 (local address 127.16.79.126:32973)
0619 06:44:37.482490 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:37.482501 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:37.482539 (+ 38us) server_negotiation.cc:407] Connection header received
0619 06:44:37.484181 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:37.484210 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:37.484597 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:37.484940 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:37.486707 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:37.487973 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:37.489696 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:37.490353 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:37.490480 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:37.495155 (+ 4675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:37.495162 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:37.495176 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:37.495212 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:37.498302 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:37.498984 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:37.498993 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:37.499008 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:37.499176 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:37.499549 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:37.499554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:37.499557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:37.500058 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:37.500405 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:37.500817 (+ 412us) server_negotiation.cc:299] Negotiation successful
0619 06:44:37.501216 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":371,"threads_started":1}
I20260619 06:44:37.503616 19686 raft_consensus.cc:348] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35d88b6b43f4f7f9c8154b5b6c26003" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 32973 } }
I20260619 06:44:37.504017 19686 raft_consensus.cc:374] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:37.504154 19686 raft_consensus.cc:729] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c35d88b6b43f4f7f9c8154b5b6c26003, State: Initialized, Role: FOLLOWER
I20260619 06:44:37.504820 19686 consensus_queue.cc:260] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35d88b6b43f4f7f9c8154b5b6c26003" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 32973 } }
I20260619 06:44:37.505091 19686 raft_consensus.cc:388] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:44:37.505246 19686 raft_consensus.cc:482] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:44:37.505411 19686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:37.505745 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:44:37.507472 19686 raft_consensus.cc:504] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35d88b6b43f4f7f9c8154b5b6c26003" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 32973 } }
I20260619 06:44:37.507926 19686 leader_election.cc:302] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c35d88b6b43f4f7f9c8154b5b6c26003; no voters:
I20260619 06:44:37.508565 19686 leader_election.cc:288] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:44:37.508673 19691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:37.509951 19691 raft_consensus.cc:686] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [term 1 LEADER]: Becoming Leader. State: Replica: c35d88b6b43f4f7f9c8154b5b6c26003, State: Running, Role: LEADER
I20260619 06:44:37.510533 19691 consensus_queue.cc:237] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35d88b6b43f4f7f9c8154b5b6c26003" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 32973 } }
I20260619 06:44:37.510867 19686 sys_catalog.cc:564] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:44:37.513470 19693 sys_catalog.cc:455] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c35d88b6b43f4f7f9c8154b5b6c26003. Latest consensus state: current_term: 1 leader_uuid: "c35d88b6b43f4f7f9c8154b5b6c26003" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35d88b6b43f4f7f9c8154b5b6c26003" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 32973 } } }
I20260619 06:44:37.513890 19693 sys_catalog.cc:458] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:37.514549 19692 sys_catalog.cc:455] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c35d88b6b43f4f7f9c8154b5b6c26003" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35d88b6b43f4f7f9c8154b5b6c26003" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 32973 } } }
I20260619 06:44:37.514865 19692 sys_catalog.cc:458] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003 [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:37.517066 19698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:44:37.523392 19698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:44:37.535164 19698 catalog_manager.cc:1347] Generated new cluster ID: d1e6fc5178004ddf90104db1364e0da3
I20260619 06:44:37.535317 19698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:44:37.547487 19698 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:44:37.549139 19698 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:44:37.551975 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:37.542894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40354 (local address 127.16.79.126:32973)
0619 06:44:37.543080 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:37.543086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:37.543662 (+ 576us) server_negotiation.cc:407] Connection header received
0619 06:44:37.543881 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:37.543888 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:37.543972 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:37.544079 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:37.545423 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:37.546382 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:37.548042 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:37.548773 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:37.548917 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:37.550155 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:37.550162 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:37.550167 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:37.551014 (+ 847us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:37.551109 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:37.551143 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:37.551252 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:37.551436 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 06:44:37.551573 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":60}
I20260619 06:44:37.555757 19698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c35d88b6b43f4f7f9c8154b5b6c26003: Generated new TSK 0
I20260619 06:44:37.556761 19698 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:44:37.570325 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 19619
I20260619 06:44:37.571691 19681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:37.659071 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 19619
2026-06-19T06:44:37Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1864 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:44:37 dist-test-slave-5w2x krb5kdc[19721](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:44:37 dist-test-slave-5w2x krb5kdc[19721](info): set up 2 sockets
Jun 19 06:44:37 dist-test-slave-5w2x krb5kdc[19721](info): commencing operation
krb5kdc: starting...
W20260619 06:44:38.736497 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:44:38 dist-test-slave-5w2x krb5kdc[19721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:44:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:44:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:44:38.927013 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:45299
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:45299
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:39.246830 19734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:39.247143 19734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:39.247305 19734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:39.254559 19734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:44:39.254666 19734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:39.254757 19734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:39.254838 19734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:44:39.254901 19734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:44:39.267542 19734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:45299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:45299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:39.268939 19734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:39.270212 19734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:39.281802 19739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:39.281744 19742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:39.281826 19740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:39.282864 19734 server_base.cc:1034] running on GCE node
I20260619 06:44:39.283690 19734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:39.285271 19734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:39.286482 19734 hybrid_clock.cc:648] HybridClock initialized: now 1781851479286434 us; error 36 us; skew 500 ppm
I20260619 06:44:39.286829 19734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:39 dist-test-slave-5w2x krb5kdc[19721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851479, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:39.290052 19734 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:44:39.291608 19734 webserver.cc:460] Webserver started at http://127.16.79.126:40737/ using document root <none> and password file <none>
I20260619 06:44:39.292382 19734 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:39.292522 19734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:39.292827 19734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:39.295753 19734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "fb2c7f8bd4b24eb99be35a45ab826a6a"
format_stamp: "Formatted at 2026-06-19 06:44:39 on dist-test-slave-5w2x"
server_key: "15695e10786d0feb8d3651ae9faa21ba"
server_key_iv: "963c4f0a5f55f51b00b53d6f992ac091"
server_key_version: "encryptionkey@0"
I20260619 06:44:39.296581 19734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "fb2c7f8bd4b24eb99be35a45ab826a6a"
format_stamp: "Formatted at 2026-06-19 06:44:39 on dist-test-slave-5w2x"
server_key: "15695e10786d0feb8d3651ae9faa21ba"
server_key_iv: "963c4f0a5f55f51b00b53d6f992ac091"
server_key_version: "encryptionkey@0"
I20260619 06:44:39.302311 19734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260619 06:44:39.306044 19749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:39.307471 19734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:44:39.307680 19734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "fb2c7f8bd4b24eb99be35a45ab826a6a"
format_stamp: "Formatted at 2026-06-19 06:44:39 on dist-test-slave-5w2x"
server_key: "15695e10786d0feb8d3651ae9faa21ba"
server_key_iv: "963c4f0a5f55f51b00b53d6f992ac091"
server_key_version: "encryptionkey@0"
I20260619 06:44:39.307858 19734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:39.328011 19734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:39.332494 19734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:39.332796 19734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:39.351565 19734 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:45299
I20260619 06:44:39.351588 19800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:45299 every 8 connection(s)
I20260619 06:44:39.353024 19734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:44:39.355587 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 19734
I20260619 06:44:39.355799 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:44:39.356632 16701 external_mini_cluster.cc:1428] Setting key 3f43743a524725c1a71c7b84b5800b90
I20260619 06:44:39.357466 19801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 06:44:39 dist-test-slave-5w2x krb5kdc[19721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:39.370215 19801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a: Bootstrap starting.
I20260619 06:44:39.373805 19801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:39.374900 19801 log.cc:824] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:39.378006 19801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a: No bootstrap required, opened a new log
I20260619 06:44:39.379124 19804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:39.358744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52632 (local address 127.16.79.126:45299)
0619 06:44:39.359495 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:39.359506 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:39.359546 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:44:39.361123 (+ 1577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:39.361145 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:39.361569 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:39.362017 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:39.363371 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:39.364772 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:39.366059 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:39.366764 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:39.367005 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:39.371271 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:39.371279 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:39.371294 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:39.371337 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:39.374688 (+ 3351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:39.375535 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:39.375542 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:39.375549 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:39.375730 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:39.376188 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:39.376195 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:39.376198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:39.376783 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:39.377115 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:39.377574 (+ 459us) server_negotiation.cc:299] Negotiation successful
0619 06:44:39.378034 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":337,"threads_started":1}
I20260619 06:44:39.385828 19801 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2c7f8bd4b24eb99be35a45ab826a6a" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45299 } }
I20260619 06:44:39.386268 19801 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:39.386411 19801 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2c7f8bd4b24eb99be35a45ab826a6a, State: Initialized, Role: FOLLOWER
I20260619 06:44:39.387002 19801 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2c7f8bd4b24eb99be35a45ab826a6a" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45299 } }
I20260619 06:44:39.387251 19801 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:44:39.387418 19801 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:44:39.387602 19801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:39.390185 19801 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2c7f8bd4b24eb99be35a45ab826a6a" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45299 } }
I20260619 06:44:39.390702 19801 leader_election.cc:302] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb2c7f8bd4b24eb99be35a45ab826a6a; no voters:
I20260619 06:44:39.391323 19801 leader_election.cc:288] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:44:39.391469 19806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:39.392606 19806 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [term 1 LEADER]: Becoming Leader. State: Replica: fb2c7f8bd4b24eb99be35a45ab826a6a, State: Running, Role: LEADER
I20260619 06:44:39.393157 19806 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2c7f8bd4b24eb99be35a45ab826a6a" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45299 } }
I20260619 06:44:39.393778 19801 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:44:39.398355 19807 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb2c7f8bd4b24eb99be35a45ab826a6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2c7f8bd4b24eb99be35a45ab826a6a" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45299 } } }
I20260619 06:44:39.398777 19807 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:39.398622 19808 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb2c7f8bd4b24eb99be35a45ab826a6a. Latest consensus state: current_term: 1 leader_uuid: "fb2c7f8bd4b24eb99be35a45ab826a6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb2c7f8bd4b24eb99be35a45ab826a6a" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45299 } } }
I20260619 06:44:39.399572 19808 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:39.400760 19813 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:44:39.406327 19813 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:44:39.415584 19813 catalog_manager.cc:1347] Generated new cluster ID: 5e9b8572a38c48e4b32aee1136ada7b5
I20260619 06:44:39.415700 19813 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:44:39.448655 19813 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:44:39.450012 19813 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:44:39.461229 19813 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb2c7f8bd4b24eb99be35a45ab826a6a: Generated new TSK 0
I20260619 06:44:39.462133 19813 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:44:39.516531 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:45299
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:39.861948 19827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:39.862274 19827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:39.862447 19827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:39.869756 19827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:39.869920 19827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:39.870097 19827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:44:39.880668 19827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:45299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:39.882006 19827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:39.883283 19827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:39.895989 19833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:39.896162 19832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:39.896359 19835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:39.897372 19827 server_base.cc:1034] running on GCE node
I20260619 06:44:39.898047 19827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:39.898988 19827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:39.900223 19827 hybrid_clock.cc:648] HybridClock initialized: now 1781851479900174 us; error 34 us; skew 500 ppm
I20260619 06:44:39.900545 19827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:39 dist-test-slave-5w2x krb5kdc[19721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851479, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:39.903771 19827 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:44:39.905296 19827 webserver.cc:460] Webserver started at http://127.16.79.65:43159/ using document root <none> and password file <none>
I20260619 06:44:39.906050 19827 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:39.906188 19827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:39.906491 19827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:39.909420 19827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "5418e23296d14b14a363ed9a8a1ec290"
format_stamp: "Formatted at 2026-06-19 06:44:39 on dist-test-slave-5w2x"
server_key: "f1fcc2d07431bb51bd313ac3e4b1c65c"
server_key_iv: "3d5b75f4fdb10f1f4b3d2fb68514002c"
server_key_version: "encryptionkey@0"
I20260619 06:44:39.910203 19827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "5418e23296d14b14a363ed9a8a1ec290"
format_stamp: "Formatted at 2026-06-19 06:44:39 on dist-test-slave-5w2x"
server_key: "f1fcc2d07431bb51bd313ac3e4b1c65c"
server_key_iv: "3d5b75f4fdb10f1f4b3d2fb68514002c"
server_key_version: "encryptionkey@0"
I20260619 06:44:39.915369 19827 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 06:44:39.918916 19842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:39.920233 19827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:44:39.920398 19827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "5418e23296d14b14a363ed9a8a1ec290"
format_stamp: "Formatted at 2026-06-19 06:44:39 on dist-test-slave-5w2x"
server_key: "f1fcc2d07431bb51bd313ac3e4b1c65c"
server_key_iv: "3d5b75f4fdb10f1f4b3d2fb68514002c"
server_key_version: "encryptionkey@0"
I20260619 06:44:39.920593 19827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:39.949069 19827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:39.953500 19827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:39.953807 19827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:39.954772 19827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:39.956180 19827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:39.956287 19827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:39.956426 19827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:39.956496 19827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:39.990831 19827 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:43387
I20260619 06:44:39.990880 19954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:43387 every 8 connection(s)
I20260619 06:44:39.992161 19827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:44:39.997722 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 19827
I20260619 06:44:39.997928 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:44:39.998739 16701 external_mini_cluster.cc:1428] Setting key dbd6e8fa5e1b917b971b10e9ce9bec76
Jun 19 06:44:40 dist-test-slave-5w2x krb5kdc[19721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851479, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:40.014688 19957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:39.994849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:33559 (local address 127.16.79.126:45299)
0619 06:44:39.995339 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:39.995343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:39.996367 (+ 1024us) server_negotiation.cc:407] Connection header received
0619 06:44:39.998900 (+ 2533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:39.998905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:39.998996 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:39.999104 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:40.000854 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:40.001936 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:40.003683 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:40.004452 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:40.004624 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:40.008524 (+ 3900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:40.008529 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:40.008532 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:40.008568 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:40.011249 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:40.012260 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:40.012266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:40.012271 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:40.012431 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:40.012902 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:40.012907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:40.012910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:40.013184 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:40.013340 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:40.014073 (+ 733us) server_negotiation.cc:299] Negotiation successful
0619 06:44:40.014297 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":262,"threads_started":1}
I20260619 06:44:40.015684 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:39.995481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:45299 (local address 127.16.79.65:33559)
0619 06:44:39.996195 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:39.996258 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:39.998483 (+ 2225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:39.999409 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:39.999426 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:39.999947 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:40.000617 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:40.000635 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:40.002079 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:40.002083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:40.003521 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:40.003534 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:40.004756 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:40.004762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:40.004987 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:40.005786 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:40.005812 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:40.008258 (+ 2446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:40.011471 (+ 3213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:40.011481 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:40.011516 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:40.012080 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:40.012543 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:40.012548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:40.012551 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:40.012778 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:40.013314 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:40.013334 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:40.013706 (+ 372us) client_negotiation.cc:769] Sending connection context
0619 06:44:40.013976 (+ 270us) client_negotiation.cc:240] Negotiation successful
0619 06:44:40.014529 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":258,"threads_started":1}
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
I20260619 06:44:40.017573 19955 heartbeater.cc:347] Connected to a master server at 127.16.79.126:45299
I20260619 06:44:40.017944 19955 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:40.018749 19955 heartbeater.cc:511] Master 127.16.79.126:45299 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
I20260619 06:44:40.021055 19766 ts_manager.cc:194] Registered new tserver with Master: 5418e23296d14b14a363ed9a8a1ec290 (127.16.79.65:43387)
I20260619 06:44:40.022498 19766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:33559
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:44:40.043222 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:45299
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:40.362648 19961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:40.362982 19961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:40.363152 19961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:40.370607 19961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:40.370779 19961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:40.370965 19961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:44:40.380947 19961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:45299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:40.382294 19961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:40.383605 19961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:40.397043 19967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:40.396991 19969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:40.397080 19966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:40.397871 19961 server_base.cc:1034] running on GCE node
I20260619 06:44:40.398471 19961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:40.399435 19961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:40.400671 19961 hybrid_clock.cc:648] HybridClock initialized: now 1781851480400617 us; error 73 us; skew 500 ppm
I20260619 06:44:40.401003 19961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:40 dist-test-slave-5w2x krb5kdc[19721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851480, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:40.404245 19961 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:44:40.405686 19961 webserver.cc:460] Webserver started at http://127.16.79.66:39365/ using document root <none> and password file <none>
I20260619 06:44:40.406400 19961 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:40.406548 19961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:40.406876 19961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:40.410014 19961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "86965252327c45aaa2fa00aafff6d9cc"
format_stamp: "Formatted at 2026-06-19 06:44:40 on dist-test-slave-5w2x"
server_key: "788ae97a7d109c26b7be61ae51bca29c"
server_key_iv: "c11a7fb85fa4db650226fc292b38cdda"
server_key_version: "encryptionkey@0"
I20260619 06:44:40.410808 19961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "86965252327c45aaa2fa00aafff6d9cc"
format_stamp: "Formatted at 2026-06-19 06:44:40 on dist-test-slave-5w2x"
server_key: "788ae97a7d109c26b7be61ae51bca29c"
server_key_iv: "c11a7fb85fa4db650226fc292b38cdda"
server_key_version: "encryptionkey@0"
I20260619 06:44:40.415982 19961 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260619 06:44:40.419317 19976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:40.420645 19961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:44:40.420819 19961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "86965252327c45aaa2fa00aafff6d9cc"
format_stamp: "Formatted at 2026-06-19 06:44:40 on dist-test-slave-5w2x"
server_key: "788ae97a7d109c26b7be61ae51bca29c"
server_key_iv: "c11a7fb85fa4db650226fc292b38cdda"
server_key_version: "encryptionkey@0"
I20260619 06:44:40.421017 19961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:40.433267 19961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:40.437462 19961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:40.437788 19961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:40.438789 19961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:40.440347 19961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:40.440454 19961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:40.440626 19961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:40.440694 19961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:40.473870 19961 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:34057
I20260619 06:44:40.473917 20088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:34057 every 8 connection(s)
I20260619 06:44:40.475199 19961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:44:40.481503 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 19961
I20260619 06:44:40.481676 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:44:40.482115 16701 external_mini_cluster.cc:1428] Setting key 52a0c350573ab60c9d944b847b9688b6
Jun 19 06:44:40 dist-test-slave-5w2x krb5kdc[19721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851480, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:40.495425 19957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:40.477877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:56159 (local address 127.16.79.126:45299)
0619 06:44:40.478151 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:40.478158 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:40.479297 (+ 1139us) server_negotiation.cc:407] Connection header received
0619 06:44:40.481080 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:40.481085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:40.481170 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:40.481293 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:40.482969 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:40.483683 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:40.485299 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:40.485723 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:40.485862 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:40.489577 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:40.489583 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:40.489586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:40.489618 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:40.492294 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:40.493038 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:40.493043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:40.493053 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:40.493202 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:40.493634 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:40.493646 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:40.493648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:40.493988 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:40.494153 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:40.494837 (+ 684us) server_negotiation.cc:299] Negotiation successful
0619 06:44:40.495082 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 06:44:40.496564 20091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:40.478454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:45299 (local address 127.16.79.66:56159)
0619 06:44:40.479122 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:40.479164 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:40.480823 (+ 1659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:40.481548 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:40.481557 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:40.482148 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:40.482758 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:40.482797 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:40.483861 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:40.483865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:40.485143 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:40.485159 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:40.485956 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:40.485963 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:40.486204 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:40.486906 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:40.486929 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:40.489337 (+ 2408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:40.492433 (+ 3096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:40.492443 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:40.492461 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:40.492898 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:40.493313 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:40.493318 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:40.493322 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:40.493516 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:40.494113 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:40.494123 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:40.494474 (+ 351us) client_negotiation.cc:769] Sending connection context
0619 06:44:40.494738 (+ 264us) client_negotiation.cc:240] Negotiation successful
0619 06:44:40.495303 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":259,"threads_started":1}
I20260619 06:44:40.498055 20089 heartbeater.cc:347] Connected to a master server at 127.16.79.126:45299
I20260619 06:44:40.498425 20089 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:40.499413 20089 heartbeater.cc:511] Master 127.16.79.126:45299 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
I20260619 06:44:40.501360 19766 ts_manager.cc:194] Registered new tserver with Master: 86965252327c45aaa2fa00aafff6d9cc (127.16.79.66:34057)
I20260619 06:44:40.502345 19766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:56159
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:44:40.525511 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:45299
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:40.850694 20094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:40.850991 20094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:40.851158 20094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:40.858342 20094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:40.858495 20094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:40.858690 20094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:44:40.868348 20094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:45299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:40.869652 20094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:40.870968 20094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:40.884061 20102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:40.884095 20099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:40.884025 20100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:40.885118 20094 server_base.cc:1034] running on GCE node
I20260619 06:44:40.885736 20094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:40.886643 20094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:40.887835 20094 hybrid_clock.cc:648] HybridClock initialized: now 1781851480887779 us; error 59 us; skew 500 ppm
I20260619 06:44:40.888181 20094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:40 dist-test-slave-5w2x krb5kdc[19721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851480, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:40.891211 20094 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:44:40.892701 20094 webserver.cc:460] Webserver started at http://127.16.79.67:44655/ using document root <none> and password file <none>
I20260619 06:44:40.893417 20094 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:40.893560 20094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:40.893913 20094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:40.896868 20094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "df7d4a959532413aa0e004afe067b7a5"
format_stamp: "Formatted at 2026-06-19 06:44:40 on dist-test-slave-5w2x"
server_key: "88dc9b3db6bff2be67d20e8414b46b53"
server_key_iv: "5ac1c193a7df6183e3756e04fc3c8b7a"
server_key_version: "encryptionkey@0"
I20260619 06:44:40.897624 20094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "df7d4a959532413aa0e004afe067b7a5"
format_stamp: "Formatted at 2026-06-19 06:44:40 on dist-test-slave-5w2x"
server_key: "88dc9b3db6bff2be67d20e8414b46b53"
server_key_iv: "5ac1c193a7df6183e3756e04fc3c8b7a"
server_key_version: "encryptionkey@0"
I20260619 06:44:40.902612 20094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260619 06:44:40.906212 20109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:40.907482 20094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:44:40.907649 20094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "df7d4a959532413aa0e004afe067b7a5"
format_stamp: "Formatted at 2026-06-19 06:44:40 on dist-test-slave-5w2x"
server_key: "88dc9b3db6bff2be67d20e8414b46b53"
server_key_iv: "5ac1c193a7df6183e3756e04fc3c8b7a"
server_key_version: "encryptionkey@0"
I20260619 06:44:40.907846 20094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:40.928591 20094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:40.932821 20094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:40.933118 20094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:40.934087 20094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:40.935370 20094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:40.935477 20094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:40.935623 20094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:40.935695 20094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:40.969009 20094 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:32817
I20260619 06:44:40.969096 20221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:32817 every 8 connection(s)
I20260619 06:44:40.970299 20094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:44:40.973361 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 20094
I20260619 06:44:40.973533 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:44:40.974030 16701 external_mini_cluster.cc:1428] Setting key a2f6b1179c95d8944df824ae3e9e4179
Jun 19 06:44:40 dist-test-slave-5w2x krb5kdc[19721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851480, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:40.989579 19957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:40.972903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:58695 (local address 127.16.79.126:45299)
0619 06:44:40.973113 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:40.973119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:40.974478 (+ 1359us) server_negotiation.cc:407] Connection header received
0619 06:44:40.976505 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:40.976509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:40.976571 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:40.976688 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:40.978186 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:40.978908 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:40.980449 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:40.980837 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:40.980926 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:40.984015 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:40.984019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:40.984022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:40.984052 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:40.986559 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:40.987285 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:40.987292 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:40.987296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:40.987434 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:40.987924 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:40.987927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:40.987929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:40.988239 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:40.988345 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:40.989169 (+ 824us) server_negotiation.cc:299] Negotiation successful
0619 06:44:40.989333 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 06:44:40.990661 20224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:40.973434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:45299 (local address 127.16.79.67:58695)
0619 06:44:40.974302 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:40.974347 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:40.976200 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:40.976972 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:40.976980 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:40.977466 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:40.978007 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:40.978024 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:40.979038 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:40.979042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:40.980315 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:40.980326 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:40.981008 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:40.981013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:40.981199 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:40.981797 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:40.981817 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:40.983840 (+ 2023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:40.986752 (+ 2912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:40.986758 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:40.986769 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:40.987155 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:40.987550 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:40.987556 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:40.987564 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:40.987766 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:40.988347 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:40.988358 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:40.988748 (+ 390us) client_negotiation.cc:769] Sending connection context
0619 06:44:40.989058 (+ 310us) client_negotiation.cc:240] Negotiation successful
0619 06:44:40.989432 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":370,"threads_started":1}
I20260619 06:44:40.992019 20222 heartbeater.cc:347] Connected to a master server at 127.16.79.126:45299
I20260619 06:44:40.992331 20222 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:40.993005 20222 heartbeater.cc:511] Master 127.16.79.126:45299 requested a full tablet report, sending...
I20260619 06:44:40.994441 19766 ts_manager.cc:194] Registered new tserver with Master: df7d4a959532413aa0e004afe067b7a5 (127.16.79.67:32817)
I20260619 06:44:40.995254 19766 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:58695
I20260619 06:44:41.001101 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 06:44:41 dist-test-slave-5w2x krb5kdc[19721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851481, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM: I20260619 06:44:41.025712 19955 heartbeater.cc:503] Master 127.16.79.126:45299 was elected leader, sending a full tablet report...
Jun 19 06:44:41 dist-test-slave-5w2x krb5kdc[19721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851481, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:41.047147 19957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:41.033623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52636 (local address 127.16.79.126:45299)
0619 06:44:41.033782 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:41.033786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:41.034270 (+ 484us) server_negotiation.cc:407] Connection header received
0619 06:44:41.034476 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:41.034487 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:41.034549 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:41.034654 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:41.035956 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:41.036753 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:41.038038 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:41.038445 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:41.038580 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:41.042238 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:41.042243 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:41.042246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:41.042276 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:41.044634 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:41.045279 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:41.045291 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:41.045294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:41.045426 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:41.045810 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:41.045817 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:41.045820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:41.046107 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:41.046355 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:41.046618 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 06:44:41.046825 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260619 06:44:41.047646 19766 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52636
I20260619 06:44:41.049597 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 19827
Jun 19 06:44:41 dist-test-slave-5w2x krb5kdc[19721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851481, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:44:41.064056 20233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:41.050438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44626 (local address 127.16.79.65:43387)
0619 06:44:41.051074 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:41.051081 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:41.051104 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:44:41.051157 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:41.051163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:41.051375 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:41.051505 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:41.052601 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:41.053552 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:41.054767 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:41.055343 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:41.055441 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:41.059119 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:41.059132 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:41.059146 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:41.059182 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:41.061804 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:41.062375 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:41.062381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:41.062383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:41.062511 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:41.062811 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:41.062815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:41.062817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:41.063143 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:41.063361 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:41.063588 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 06:44:41.063776 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":330,"threads_started":1}
W20260619 06:44:41.065395 19950 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44626
W20260619 06:44:41.066512 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 19827: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:44:41.066704 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 19827
I20260619 06:44:41.081715 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 19961
Jun 19 06:44:41 dist-test-slave-5w2x krb5kdc[19721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851481, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:44:41.097026 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:41.082489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59896 (local address 127.16.79.66:34057)
0619 06:44:41.083062 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:41.083068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:41.083092 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:44:41.083149 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:41.083155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:41.083355 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:41.083479 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:41.084642 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:41.085608 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:41.086735 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:41.087262 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:41.087358 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:41.091279 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:41.091290 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:41.091304 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:41.091335 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:41.094172 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:41.094871 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:41.094878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:41.094880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:41.094997 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:41.095351 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:41.095356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:41.095359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:41.095734 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:41.096046 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:41.096414 (+ 368us) server_negotiation.cc:299] Negotiation successful
0619 06:44:41.096670 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":302,"threads_started":1}
W20260619 06:44:41.098198 20084 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59896
W20260619 06:44:41.099236 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 19961: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:44:41.099375 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 19961
I20260619 06:44:41.114959 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 20094
Jun 19 06:44:41 dist-test-slave-5w2x krb5kdc[19721](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851481, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:44:41.131719 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:41.115989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40318 (local address 127.16.79.67:32817)
0619 06:44:41.116665 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:41.116672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:41.116699 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:44:41.116756 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:41.116762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:41.116985 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:41.117174 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:41.118526 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:41.119553 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:41.120904 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:41.121471 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:41.121612 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:41.126111 (+ 4499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:41.126124 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:41.126140 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:41.126179 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:41.128954 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:41.129635 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:41.129643 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:41.129646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:41.129765 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:41.130119 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:41.130126 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:41.130129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:41.130461 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:41.130769 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:41.131098 (+ 329us) server_negotiation.cc:299] Negotiation successful
0619 06:44:41.131371 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":398,"threads_started":1}
W20260619 06:44:41.133093 20217 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40318
W20260619 06:44:41.134334 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 20094: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:44:41.134503 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 20094
I20260619 06:44:41.149636 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 19734
I20260619 06:44:41.150767 19796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:41.253747 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 19734
2026-06-19T06:44:41Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3602 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:44:41 dist-test-slave-5w2x krb5kdc[20239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:44:41 dist-test-slave-5w2x krb5kdc[20239](info): set up 2 sockets
Jun 19 06:44:41 dist-test-slave-5w2x krb5kdc[20239](info): commencing operation
krb5kdc: starting...
W20260619 06:44:42.340602 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.004s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:44:42 dist-test-slave-5w2x krb5kdc[20239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:44:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:44:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:44:42.526845 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:39087
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:39087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:42.842561 20252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:42.842926 20252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:42.843092 20252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:42.850363 20252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:44:42.850526 20252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:42.850636 20252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:42.850745 20252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:44:42.850842 20252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:44:42.860632 20252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:39087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:39087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:42.861953 20252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:42.863303 20252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:42.874972 20257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:42.874987 20260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:42.875444 20258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:42.875952 20252 server_base.cc:1034] running on GCE node
I20260619 06:44:42.876840 20252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:42.878213 20252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:42.879422 20252 hybrid_clock.cc:648] HybridClock initialized: now 1781851482879378 us; error 39 us; skew 500 ppm
I20260619 06:44:42.879791 20252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:42 dist-test-slave-5w2x krb5kdc[20239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851482, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:42.886274 20252 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:44:42.887715 20252 webserver.cc:460] Webserver started at http://127.16.79.126:41569/ using document root <none> and password file <none>
I20260619 06:44:42.888437 20252 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:42.888537 20252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:42.888785 20252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:42.891608 20252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "d6c467780c9b43e2b0433aed17c739c4"
format_stamp: "Formatted at 2026-06-19 06:44:42 on dist-test-slave-5w2x"
server_key: "436abf1bb4ef9d6e5721043464ecba39"
server_key_iv: "9c3e4b6d0f2cc2948dacee550d0d6343"
server_key_version: "encryptionkey@0"
I20260619 06:44:42.892411 20252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "d6c467780c9b43e2b0433aed17c739c4"
format_stamp: "Formatted at 2026-06-19 06:44:42 on dist-test-slave-5w2x"
server_key: "436abf1bb4ef9d6e5721043464ecba39"
server_key_iv: "9c3e4b6d0f2cc2948dacee550d0d6343"
server_key_version: "encryptionkey@0"
I20260619 06:44:42.897388 20252 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 06:44:42.900640 20267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:42.901964 20252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:44:42.902099 20252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "d6c467780c9b43e2b0433aed17c739c4"
format_stamp: "Formatted at 2026-06-19 06:44:42 on dist-test-slave-5w2x"
server_key: "436abf1bb4ef9d6e5721043464ecba39"
server_key_iv: "9c3e4b6d0f2cc2948dacee550d0d6343"
server_key_version: "encryptionkey@0"
I20260619 06:44:42.902258 20252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:42.931459 20252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:42.935491 20252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:42.935719 20252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:42.954501 20252 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:39087
I20260619 06:44:42.954535 20318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:39087 every 8 connection(s)
I20260619 06:44:42.955842 20252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:44:42.959525 20319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:42.964468 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 20252
I20260619 06:44:42.964697 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:44:42.965319 16701 external_mini_cluster.cc:1428] Setting key 694095319ec5b7447d0b2e1e4ec69013
I20260619 06:44:42.970252 20319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4: Bootstrap starting.
I20260619 06:44:42.973230 20319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:42.974201 20319 log.cc:824] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:42.976979 20319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4: No bootstrap required, opened a new log
Jun 19 06:44:42 dist-test-slave-5w2x krb5kdc[20239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:42.984712 20319 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6c467780c9b43e2b0433aed17c739c4" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39087 } }
I20260619 06:44:42.985117 20319 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:42.985245 20319 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6c467780c9b43e2b0433aed17c739c4, State: Initialized, Role: FOLLOWER
I20260619 06:44:42.985957 20319 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6c467780c9b43e2b0433aed17c739c4" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39087 } }
I20260619 06:44:42.986222 20319 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:44:42.986377 20319 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:44:42.986581 20319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:42.987152 20322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:42.967308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.16.79.126:39087)
0619 06:44:42.968076 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:42.968090 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:42.968170 (+ 80us) server_negotiation.cc:407] Connection header received
0619 06:44:42.970010 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:42.970032 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:42.970463 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:42.970835 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:42.972306 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:42.973597 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:42.974903 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:42.975578 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:42.975765 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:42.979617 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:42.979625 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:42.979642 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:42.979678 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:42.982846 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:42.983582 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:42.983589 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:42.983595 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:42.983768 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:42.984183 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:42.984188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:42.984193 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:42.984721 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:42.985042 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:42.985463 (+ 421us) server_negotiation.cc:299] Negotiation successful
0619 06:44:42.985888 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":323,"threads_started":1}
I20260619 06:44:42.989061 20319 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6c467780c9b43e2b0433aed17c739c4" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39087 } }
I20260619 06:44:42.989565 20319 leader_election.cc:302] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6c467780c9b43e2b0433aed17c739c4; no voters:
I20260619 06:44:42.990126 20319 leader_election.cc:288] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:44:42.990274 20324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:42.991060 20324 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [term 1 LEADER]: Becoming Leader. State: Replica: d6c467780c9b43e2b0433aed17c739c4, State: Running, Role: LEADER
I20260619 06:44:42.991645 20324 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6c467780c9b43e2b0433aed17c739c4" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39087 } }
I20260619 06:44:42.992295 20319 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:44:42.994551 20325 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6c467780c9b43e2b0433aed17c739c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6c467780c9b43e2b0433aed17c739c4" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39087 } } }
I20260619 06:44:42.994560 20326 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6c467780c9b43e2b0433aed17c739c4. Latest consensus state: current_term: 1 leader_uuid: "d6c467780c9b43e2b0433aed17c739c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6c467780c9b43e2b0433aed17c739c4" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39087 } } }
I20260619 06:44:42.995118 20325 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:42.995116 20326 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4 [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:42.998903 20333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:44:43.003866 20333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:44:43.012915 20333 catalog_manager.cc:1347] Generated new cluster ID: 1a7aadc7546f4c54aeecc86f9c350ba9
I20260619 06:44:43.013094 20333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:44:43.024278 20333 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:44:43.025743 20333 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:44:43.033072 20333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6c467780c9b43e2b0433aed17c739c4: Generated new TSK 0
I20260619 06:44:43.033927 20333 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:44:43.084359 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:39087
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:43.426730 20345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:43.427026 20345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:43.427122 20345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:43.434398 20345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:43.434509 20345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:43.434624 20345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:44:43.444506 20345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:43.445756 20345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:43.446960 20345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:43.459756 20353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:43.459800 20351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:43.459717 20350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:43.464890 20345 server_base.cc:1034] running on GCE node
I20260619 06:44:43.465548 20345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:43.466549 20345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:43.467788 20345 hybrid_clock.cc:648] HybridClock initialized: now 1781851483467720 us; error 64 us; skew 500 ppm
I20260619 06:44:43.468181 20345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:43 dist-test-slave-5w2x krb5kdc[20239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851483, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:43.471273 20345 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:44:43.472863 20345 webserver.cc:460] Webserver started at http://127.16.79.65:33705/ using document root <none> and password file <none>
I20260619 06:44:43.473587 20345 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:43.473730 20345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:43.474041 20345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:43.477034 20345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "8c99f81aaf0b451fbf7439916490a8c0"
format_stamp: "Formatted at 2026-06-19 06:44:43 on dist-test-slave-5w2x"
server_key: "e73b8f0ddd887b984cac308eb7727612"
server_key_iv: "babbe3cbcbe5f95d6b370b66cc485940"
server_key_version: "encryptionkey@0"
I20260619 06:44:43.477794 20345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "8c99f81aaf0b451fbf7439916490a8c0"
format_stamp: "Formatted at 2026-06-19 06:44:43 on dist-test-slave-5w2x"
server_key: "e73b8f0ddd887b984cac308eb7727612"
server_key_iv: "babbe3cbcbe5f95d6b370b66cc485940"
server_key_version: "encryptionkey@0"
I20260619 06:44:43.482887 20345 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:44:43.486531 20360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:43.488016 20345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:44:43.488238 20345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "8c99f81aaf0b451fbf7439916490a8c0"
format_stamp: "Formatted at 2026-06-19 06:44:43 on dist-test-slave-5w2x"
server_key: "e73b8f0ddd887b984cac308eb7727612"
server_key_iv: "babbe3cbcbe5f95d6b370b66cc485940"
server_key_version: "encryptionkey@0"
I20260619 06:44:43.488449 20345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:43.507174 20345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:43.511284 20345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:43.511597 20345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:43.512622 20345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:43.513988 20345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:43.514060 20345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:43.514227 20345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:43.514287 20345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:43.547844 20345 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:46041
I20260619 06:44:43.547878 20472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:46041 every 8 connection(s)
I20260619 06:44:43.549192 20345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:44:43.555966 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 20345
I20260619 06:44:43.556201 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:44:43.556804 16701 external_mini_cluster.cc:1428] Setting key cd11a527f7a251b266861aa49d585c38
Jun 19 06:44:43 dist-test-slave-5w2x krb5kdc[20239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851483, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:43.570473 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:43.551762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:50587 (local address 127.16.79.126:39087)
0619 06:44:43.552268 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:43.552273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:43.553189 (+ 916us) server_negotiation.cc:407] Connection header received
0619 06:44:43.555021 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:43.555027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:43.555127 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:43.555246 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:43.556885 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:43.557634 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:43.559181 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:43.559579 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:43.559709 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:43.563553 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:43.563563 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:43.563569 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:43.563612 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:43.567158 (+ 3546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:43.567868 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:43.567876 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:43.567883 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:43.568034 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:43.568496 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:43.568504 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:43.568510 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:43.568876 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:43.569035 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:43.569719 (+ 684us) server_negotiation.cc:299] Negotiation successful
0619 06:44:43.569970 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":259,"threads_started":1}
I20260619 06:44:43.571200 20476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:43.552295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:39087 (local address 127.16.79.65:50587)
0619 06:44:43.553015 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:43.553055 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:43.554737 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:43.555494 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:43.555503 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:43.556005 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:43.556712 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:43.556730 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:43.557791 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:43.557799 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:43.559036 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:43.559047 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:43.559819 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:43.559825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:43.560053 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:43.560768 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:43.560795 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:43.563331 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:43.567302 (+ 3971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:43.567309 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:43.567322 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:43.567736 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:43.568176 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:43.568183 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:43.568191 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:43.568369 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:43.569010 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:43.569016 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:43.569323 (+ 307us) client_negotiation.cc:769] Sending connection context
0619 06:44:43.569559 (+ 236us) client_negotiation.cc:240] Negotiation successful
0619 06:44:43.569932 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":290,"threads_started":1}
I20260619 06:44:43.572782 20473 heartbeater.cc:347] Connected to a master server at 127.16.79.126:39087
I20260619 06:44:43.573086 20473 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:43.573776 20473 heartbeater.cc:511] Master 127.16.79.126:39087 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
I20260619 06:44:43.575845 20284 ts_manager.cc:194] Registered new tserver with Master: 8c99f81aaf0b451fbf7439916490a8c0 (127.16.79.65:46041)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
I20260619 06:44:43.577517 20284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:50587
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:44:43.599685 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:39087
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:43.924962 20479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:43.925248 20479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:43.925345 20479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:43.932821 20479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:43.932943 20479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:43.933061 20479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:44:43.943109 20479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:43.944380 20479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:43.945643 20479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:43.960427 20487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:43.960433 20485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:43.961112 20484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:43.961381 20479 server_base.cc:1034] running on GCE node
I20260619 06:44:43.962054 20479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:43.963052 20479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:43.964246 20479 hybrid_clock.cc:648] HybridClock initialized: now 1781851483964181 us; error 59 us; skew 500 ppm
I20260619 06:44:43.964577 20479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:43 dist-test-slave-5w2x krb5kdc[20239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851483, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:43.967785 20479 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:44:43.972623 20479 webserver.cc:460] Webserver started at http://127.16.79.66:34771/ using document root <none> and password file <none>
I20260619 06:44:43.973346 20479 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:43.973496 20479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:43.973811 20479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:43.976763 20479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "cf08b6005a314250bcc3ba00cccbab20"
format_stamp: "Formatted at 2026-06-19 06:44:43 on dist-test-slave-5w2x"
server_key: "3c1bb6f6f49ac840d26587d11be8c5e2"
server_key_iv: "e022cd1d15441d7fc117350e3006ecbb"
server_key_version: "encryptionkey@0"
I20260619 06:44:43.977545 20479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "cf08b6005a314250bcc3ba00cccbab20"
format_stamp: "Formatted at 2026-06-19 06:44:43 on dist-test-slave-5w2x"
server_key: "3c1bb6f6f49ac840d26587d11be8c5e2"
server_key_iv: "e022cd1d15441d7fc117350e3006ecbb"
server_key_version: "encryptionkey@0"
I20260619 06:44:43.982509 20479 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:44:43.985874 20494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:43.987274 20479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:44:43.987468 20479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "cf08b6005a314250bcc3ba00cccbab20"
format_stamp: "Formatted at 2026-06-19 06:44:43 on dist-test-slave-5w2x"
server_key: "3c1bb6f6f49ac840d26587d11be8c5e2"
server_key_iv: "e022cd1d15441d7fc117350e3006ecbb"
server_key_version: "encryptionkey@0"
I20260619 06:44:43.987684 20479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:44.002137 20479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:44.006224 20479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:44.006512 20479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:44.007491 20479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:44.008845 20479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:44.008929 20479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:44.009018 20479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:44.009070 20479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:44.040658 20479 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:38107
I20260619 06:44:44.040702 20606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:38107 every 8 connection(s)
I20260619 06:44:44.041980 20479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:44:44.049044 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 20479
I20260619 06:44:44.049242 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:44:44.049784 16701 external_mini_cluster.cc:1428] Setting key 16319cdcdeb0e26af84fadfb31c2efc8
Jun 19 06:44:44 dist-test-slave-5w2x krb5kdc[20239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851483, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:44.062965 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:44.044635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:43227 (local address 127.16.79.126:39087)
0619 06:44:44.044824 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:44.044829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:44.045977 (+ 1148us) server_negotiation.cc:407] Connection header received
0619 06:44:44.047754 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:44.047759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:44.047847 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:44.047974 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:44.049868 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.050729 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.052661 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.053085 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.053178 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:44.057380 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:44.057385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:44.057389 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:44.057419 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:44.059984 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:44.060749 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:44.060754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:44.060757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:44.060883 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:44.061316 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:44.061327 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:44.061331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:44.061582 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:44.061735 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:44.062410 (+ 675us) server_negotiation.cc:299] Negotiation successful
0619 06:44:44.062635 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 06:44:44.063946 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:44.045093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:39087 (local address 127.16.79.66:43227)
0619 06:44:44.045819 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:44.045865 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:44.047489 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:44.048257 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:44.048265 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:44.048896 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:44.049674 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:44.049697 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.050886 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.050890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:44.052504 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:44.052520 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.053319 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.053329 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:44.053666 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:44.054432 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:44.054466 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:44.057142 (+ 2676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:44.060162 (+ 3020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:44.060171 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:44.060189 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:44.060614 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:44.061002 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:44.061007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:44.061011 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:44.061200 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:44.061711 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:44.061719 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:44.062064 (+ 345us) client_negotiation.cc:769] Sending connection context
0619 06:44:44.062312 (+ 248us) client_negotiation.cc:240] Negotiation successful
0619 06:44:44.062828 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":274,"threads_started":1}
I20260619 06:44:44.065454 20607 heartbeater.cc:347] Connected to a master server at 127.16.79.126:39087
I20260619 06:44:44.065806 20607 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:44.066705 20607 heartbeater.cc:511] Master 127.16.79.126:39087 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
I20260619 06:44:44.068348 20284 ts_manager.cc:194] Registered new tserver with Master: cf08b6005a314250bcc3ba00cccbab20 (127.16.79.66:38107)
I20260619 06:44:44.069202 20284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:43227
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:44:44.091528 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:39087
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:44.408924 20612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:44.409224 20612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:44.409385 20612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:44.420315 20612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:44.420426 20612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:44.420614 20612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:44:44.430323 20612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:39087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:44.431583 20612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:44.432950 20612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:44.446110 20617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:44.446087 20620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:44.446066 20618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:44.453778 20612 server_base.cc:1034] running on GCE node
I20260619 06:44:44.454464 20612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:44.455508 20612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:44.456734 20612 hybrid_clock.cc:648] HybridClock initialized: now 1781851484456671 us; error 66 us; skew 500 ppm
I20260619 06:44:44.457060 20612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:44 dist-test-slave-5w2x krb5kdc[20239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851484, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:44.460180 20612 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:44:44.461628 20612 webserver.cc:460] Webserver started at http://127.16.79.67:44341/ using document root <none> and password file <none>
I20260619 06:44:44.462345 20612 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:44.462486 20612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:44.462795 20612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:44.465788 20612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "5c5211cedf724f79ab6ce20be344e067"
format_stamp: "Formatted at 2026-06-19 06:44:44 on dist-test-slave-5w2x"
server_key: "d1700ee21077bd6ecb0541909c762f68"
server_key_iv: "d5d9ee5bd39330e4b84f465d8f0774c4"
server_key_version: "encryptionkey@0"
I20260619 06:44:44.466573 20612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "5c5211cedf724f79ab6ce20be344e067"
format_stamp: "Formatted at 2026-06-19 06:44:44 on dist-test-slave-5w2x"
server_key: "d1700ee21077bd6ecb0541909c762f68"
server_key_iv: "d5d9ee5bd39330e4b84f465d8f0774c4"
server_key_version: "encryptionkey@0"
I20260619 06:44:44.471743 20612 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:44:44.475499 20627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:44.476874 20612 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 06:44:44.477058 20612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "5c5211cedf724f79ab6ce20be344e067"
format_stamp: "Formatted at 2026-06-19 06:44:44 on dist-test-slave-5w2x"
server_key: "d1700ee21077bd6ecb0541909c762f68"
server_key_iv: "d5d9ee5bd39330e4b84f465d8f0774c4"
server_key_version: "encryptionkey@0"
I20260619 06:44:44.477275 20612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:44.511039 20612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:44.515019 20612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:44.515302 20612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:44.516295 20612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:44.517603 20612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:44.517701 20612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:44.517848 20612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:44.517918 20612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:44.551258 20612 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:45141
I20260619 06:44:44.551293 20739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:45141 every 8 connection(s)
I20260619 06:44:44.552626 20612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:44:44.559983 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 20612
I20260619 06:44:44.560228 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:44:44.560742 16701 external_mini_cluster.cc:1428] Setting key fb5a24c83a5d9744e12f6bbab65c0542
Jun 19 06:44:44 dist-test-slave-5w2x krb5kdc[20239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851484, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:44.572031 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:44.555214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:33021 (local address 127.16.79.126:39087)
0619 06:44:44.555415 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:44.555421 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:44.556643 (+ 1222us) server_negotiation.cc:407] Connection header received
0619 06:44:44.558369 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:44.558374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:44.558475 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:44.558566 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:44.560017 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.561157 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.562976 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.563559 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.563712 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:44.566895 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:44.566900 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:44.566904 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:44.566950 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:44.569341 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:44.569983 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:44.569986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:44.569988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:44.570108 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:44.570511 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:44.570515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:44.570517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:44.570793 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:44.570950 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:44.571573 (+ 623us) server_negotiation.cc:299] Negotiation successful
0619 06:44:44.571749 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:44:44.573082 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:44.555709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:39087 (local address 127.16.79.67:33021)
0619 06:44:44.556465 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:44.556526 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:44.558115 (+ 1589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:44.558767 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:44.558775 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:44.559261 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:44.559803 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:44.559818 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.561322 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.561327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:44.562822 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:44.562834 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.563776 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.563781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:44.563972 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:44.564629 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:44.564656 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:44.566706 (+ 2050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:44.569464 (+ 2758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:44.569470 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:44.569483 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:44.569875 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:44.570202 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:44.570205 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:44.570208 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:44.570383 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:44.570938 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:44.570944 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:44.571267 (+ 323us) client_negotiation.cc:769] Sending connection context
0619 06:44:44.571494 (+ 227us) client_negotiation.cc:240] Negotiation successful
0619 06:44:44.571870 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":280,"threads_started":1}
I20260619 06:44:44.574633 20740 heartbeater.cc:347] Connected to a master server at 127.16.79.126:39087
I20260619 06:44:44.574975 20740 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:44.575763 20740 heartbeater.cc:511] Master 127.16.79.126:39087 requested a full tablet report, sending...
I20260619 06:44:44.577220 20284 ts_manager.cc:194] Registered new tserver with Master: 5c5211cedf724f79ab6ce20be344e067 (127.16.79.67:45141)
I20260619 06:44:44.578006 20284 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:33021
I20260619 06:44:44.580338 20473 heartbeater.cc:503] Master 127.16.79.126:39087 was elected leader, sending a full tablet report...
I20260619 06:44:44.588094 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 06:44:44 dist-test-slave-5w2x krb5kdc[20239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 06:44:44 dist-test-slave-5w2x krb5kdc[20239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:44:44.636129 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:44.620871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60158 (local address 127.16.79.65:46041)
0619 06:44:44.621527 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:44.621540 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:44.621569 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:44:44.621630 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:44.621636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:44.621897 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:44.622028 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:44.623384 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.624552 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.625784 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.626376 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.626482 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:44.630412 (+ 3930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:44.630424 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:44.630439 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:44.630475 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:44.633249 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:44.633984 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:44.633992 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:44.633994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:44.634115 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:44.634524 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:44.634530 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:44.634533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:44.634873 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:44.635185 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:44.635499 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 06:44:44.635765 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":339,"threads_started":1}
Jun 19 06:44:44 dist-test-slave-5w2x krb5kdc[20239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:44.653857 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:44.640400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46820 (local address 127.16.79.126:39087)
0619 06:44:44.640614 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:44.640620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:44.640856 (+ 236us) server_negotiation.cc:407] Connection header received
0619 06:44:44.641036 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:44.641041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:44.641121 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:44.641224 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:44.642553 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.643274 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.644588 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.645021 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.645160 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:44.648879 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:44.648885 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:44.648888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:44.648920 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:44.651313 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:44.651921 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:44.651927 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:44.651941 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:44.652061 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:44.652455 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:44.652466 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:44.652468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:44.652790 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:44.653056 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:44.653326 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 06:44:44.653535 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260619 06:44:44.654418 20284 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46820
I20260619 06:44:44.655408 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 20345
I20260619 06:44:44.667225 20747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:44.656274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60168 (local address 127.16.79.65:46041)
0619 06:44:44.656425 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:44.656442 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:44.656768 (+ 326us) server_negotiation.cc:407] Connection header received
0619 06:44:44.656888 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:44.656891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:44.656955 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:44.657032 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:44.658104 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.658781 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.659892 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.660400 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.660504 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:44.662893 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:44.662897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:44.662900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:44.662928 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:44.665288 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:44.665835 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:44.665839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:44.665841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:44.665948 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:44.666229 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:44.666232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:44.666233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:44.666464 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:44.666627 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:44.666858 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 06:44:44.667005 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 06:44:44.667768 20468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:44.749712 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 20345
I20260619 06:44:44.765281 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 20479
Jun 19 06:44:44 dist-test-slave-5w2x krb5kdc[20239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:44:44.781145 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:44.766082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38828 (local address 127.16.79.66:38107)
0619 06:44:44.766616 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:44.766624 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:44.766648 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:44:44.766703 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:44.766709 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:44.766922 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:44.767096 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:44.768299 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.769279 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.770534 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.771106 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.771249 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:44.775648 (+ 4399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:44.775660 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:44.775674 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:44.775705 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:44.778536 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:44.779151 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:44.779157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:44.779160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:44.779269 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:44.779575 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:44.779578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:44.779580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:44.779883 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:44.780099 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:44.780477 (+ 378us) server_negotiation.cc:299] Negotiation successful
0619 06:44:44.780741 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":278,"threads_started":1}
I20260619 06:44:44.782480 20602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:44.863431 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 20479
I20260619 06:44:44.879097 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 20612
Jun 19 06:44:44 dist-test-slave-5w2x krb5kdc[20239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:44:44.895190 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:44.880115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52836 (local address 127.16.79.67:45141)
0619 06:44:44.880849 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:44.880855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:44.880883 (+ 28us) server_negotiation.cc:407] Connection header received
0619 06:44:44.880941 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:44.880947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:44.881208 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:44.881357 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:44.882665 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.883716 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.885028 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:44.885572 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:44.885668 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:44.889787 (+ 4119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:44.889798 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:44.889812 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:44.889847 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:44.892544 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:44.893224 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:44.893230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:44.893233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:44.893352 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:44.893696 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:44.893699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:44.893701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:44.894019 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:44.894233 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:44.894484 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 06:44:44.894777 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":365,"threads_started":1}
I20260619 06:44:44.896517 20735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:44.982528 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 20612
I20260619 06:44:44.998216 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 20252
I20260619 06:44:44.999385 20314 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:45.099838 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 20252
2026-06-19T06:44:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3844 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:44:45 dist-test-slave-5w2x krb5kdc[20759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:44:45 dist-test-slave-5w2x krb5kdc[20759](info): set up 2 sockets
Jun 19 06:44:45 dist-test-slave-5w2x krb5kdc[20759](info): commencing operation
krb5kdc: starting...
W20260619 06:44:46.184621 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:44:46 dist-test-slave-5w2x krb5kdc[20759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:44:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:44:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:44:46.373816 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:43505
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:43505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:46.691571 20772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:46.691948 20772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:46.692112 20772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:46.699294 20772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:44:46.699450 20772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:46.699569 20772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:46.699669 20772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:44:46.699759 20772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:44:46.709705 20772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:43505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:43505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:46.711022 20772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:46.712361 20772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:46.723796 20780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:46.723903 20777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:46.723870 20778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:46.724831 20772 server_base.cc:1034] running on GCE node
I20260619 06:44:46.725683 20772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:46.727038 20772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:46.728242 20772 hybrid_clock.cc:648] HybridClock initialized: now 1781851486728188 us; error 62 us; skew 500 ppm
I20260619 06:44:46.728590 20772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:46 dist-test-slave-5w2x krb5kdc[20759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851486, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:46.731604 20772 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:44:46.732826 20772 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:46.732959 20772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:46.733294 20772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:46.736276 20772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "97af174cf1c24f84a3086d65b02a905d"
format_stamp: "Formatted at 2026-06-19 06:44:46 on dist-test-slave-5w2x"
server_key: "2d99f3d0d6178fba30d57c179bdfc9cb"
server_key_iv: "3b150128cbffa47d0a0f5d2da9cf055f"
server_key_version: "encryptionkey@0"
I20260619 06:44:46.737061 20772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "97af174cf1c24f84a3086d65b02a905d"
format_stamp: "Formatted at 2026-06-19 06:44:46 on dist-test-slave-5w2x"
server_key: "2d99f3d0d6178fba30d57c179bdfc9cb"
server_key_iv: "3b150128cbffa47d0a0f5d2da9cf055f"
server_key_version: "encryptionkey@0"
I20260619 06:44:46.742081 20772 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260619 06:44:46.745532 20786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:46.746964 20772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:44:46.747141 20772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "97af174cf1c24f84a3086d65b02a905d"
format_stamp: "Formatted at 2026-06-19 06:44:46 on dist-test-slave-5w2x"
server_key: "2d99f3d0d6178fba30d57c179bdfc9cb"
server_key_iv: "3b150128cbffa47d0a0f5d2da9cf055f"
server_key_version: "encryptionkey@0"
I20260619 06:44:46.747356 20772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:46.766027 20772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:46.769759 20772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:46.770028 20772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:46.787743 20772 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:43505
I20260619 06:44:46.787767 20837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:43505 every 8 connection(s)
I20260619 06:44:46.788671 20772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:44:46.792174 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 20772
I20260619 06:44:46.792310 20838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:46.792400 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:44:46.793180 16701 external_mini_cluster.cc:1428] Setting key 07b3d9fafc3da5901aff563db1f5e3e1
I20260619 06:44:46.806805 20838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d: Bootstrap starting.
Jun 19 06:44:46 dist-test-slave-5w2x krb5kdc[20759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:46.810613 20838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:46.811960 20838 log.cc:824] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:46.815303 20838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d: No bootstrap required, opened a new log
I20260619 06:44:46.816941 20841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:46.795314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37976 (local address 127.16.79.126:43505)
0619 06:44:46.796113 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:46.796127 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:46.796206 (+ 79us) server_negotiation.cc:407] Connection header received
0619 06:44:46.797801 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:46.797835 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:46.798348 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:46.798818 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:46.800254 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:46.802116 (+ 1862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:46.803395 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:46.804503 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:46.804699 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:46.808510 (+ 3811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:46.808533 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:46.808557 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:46.808609 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:46.812319 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:46.813181 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:46.813187 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:46.813193 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:46.813410 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:46.813818 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:46.813823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:46.813825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:46.814510 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:46.814775 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:46.815159 (+ 384us) server_negotiation.cc:299] Negotiation successful
0619 06:44:46.815586 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":340,"threads_started":1}
I20260619 06:44:46.823722 20838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97af174cf1c24f84a3086d65b02a905d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43505 } }
I20260619 06:44:46.824062 20838 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:46.824225 20838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97af174cf1c24f84a3086d65b02a905d, State: Initialized, Role: FOLLOWER
I20260619 06:44:46.824712 20838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97af174cf1c24f84a3086d65b02a905d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43505 } }
I20260619 06:44:46.824914 20838 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:44:46.825004 20838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:44:46.825121 20838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:46.827515 20838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97af174cf1c24f84a3086d65b02a905d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43505 } }
I20260619 06:44:46.827970 20838 leader_election.cc:302] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97af174cf1c24f84a3086d65b02a905d; no voters:
I20260619 06:44:46.828553 20838 leader_election.cc:288] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:44:46.828724 20843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:46.829449 20843 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [term 1 LEADER]: Becoming Leader. State: Replica: 97af174cf1c24f84a3086d65b02a905d, State: Running, Role: LEADER
I20260619 06:44:46.829950 20843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97af174cf1c24f84a3086d65b02a905d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43505 } }
I20260619 06:44:46.830693 20838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:44:46.832948 20845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97af174cf1c24f84a3086d65b02a905d. Latest consensus state: current_term: 1 leader_uuid: "97af174cf1c24f84a3086d65b02a905d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97af174cf1c24f84a3086d65b02a905d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43505 } } }
I20260619 06:44:46.832932 20844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97af174cf1c24f84a3086d65b02a905d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97af174cf1c24f84a3086d65b02a905d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43505 } } }
I20260619 06:44:46.833513 20844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:46.833557 20845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:46.834293 20851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:44:46.841547 20851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:44:46.850390 20851 catalog_manager.cc:1347] Generated new cluster ID: f887b1cef2214d60a97da7602f700d36
I20260619 06:44:46.850569 20851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:44:46.873142 20851 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:44:46.875029 20851 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:44:46.890394 20851 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d: Generated new TSK 0
I20260619 06:44:46.891549 20851 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:44:46.939579 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43505
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:47.272902 20864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:47.273217 20864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:47.273382 20864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:47.280596 20864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:47.280745 20864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:47.280917 20864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:44:47.290438 20864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:47.291754 20864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:47.293114 20864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:47.306166 20870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:47.306264 20869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:47.306233 20872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:47.307133 20864 server_base.cc:1034] running on GCE node
I20260619 06:44:47.307745 20864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:47.308748 20864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:47.309953 20864 hybrid_clock.cc:648] HybridClock initialized: now 1781851487309873 us; error 56 us; skew 500 ppm
I20260619 06:44:47.310340 20864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:47 dist-test-slave-5w2x krb5kdc[20759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851487, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:47.313426 20864 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:44:47.314538 20864 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:47.314625 20864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:47.314865 20864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:47.317795 20864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "4c2997a5097b4633b6ce1d620a8799f2"
format_stamp: "Formatted at 2026-06-19 06:44:47 on dist-test-slave-5w2x"
server_key: "65843a9aeee98b8783c76ac14d3c5289"
server_key_iv: "0ce01ef5e99deec65b20af08464bc2f4"
server_key_version: "encryptionkey@0"
I20260619 06:44:47.318616 20864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "4c2997a5097b4633b6ce1d620a8799f2"
format_stamp: "Formatted at 2026-06-19 06:44:47 on dist-test-slave-5w2x"
server_key: "65843a9aeee98b8783c76ac14d3c5289"
server_key_iv: "0ce01ef5e99deec65b20af08464bc2f4"
server_key_version: "encryptionkey@0"
I20260619 06:44:47.323573 20864 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:44:47.326969 20878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:47.328374 20864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:44:47.328552 20864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "4c2997a5097b4633b6ce1d620a8799f2"
format_stamp: "Formatted at 2026-06-19 06:44:47 on dist-test-slave-5w2x"
server_key: "65843a9aeee98b8783c76ac14d3c5289"
server_key_iv: "0ce01ef5e99deec65b20af08464bc2f4"
server_key_version: "encryptionkey@0"
I20260619 06:44:47.328770 20864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:47.345120 20864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:47.349501 20864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:47.349716 20864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:47.350618 20864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:47.351953 20864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:47.352047 20864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:47.352128 20864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:47.352239 20864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260619 06:44:47.383862 20864 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:39825
I20260619 06:44:47.383906 20990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:39825 every 8 connection(s)
I20260619 06:44:47.384732 20864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:44:47.390547 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 20864
I20260619 06:44:47.390753 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:44:47.391309 16701 external_mini_cluster.cc:1428] Setting key 4fae10b0c4c3a1ada9ed40eb671678a3
Jun 19 06:44:47 dist-test-slave-5w2x krb5kdc[20759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851487, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:47.404086 20993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:47.387144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:38493 (local address 127.16.79.126:43505)
0619 06:44:47.387596 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:47.387601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:47.388482 (+ 881us) server_negotiation.cc:407] Connection header received
0619 06:44:47.390317 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:47.390323 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:47.390426 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:47.390557 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:47.392080 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:47.393144 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:47.394658 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:47.395064 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:47.395195 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:47.398678 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:47.398683 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:47.398686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:47.398717 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:47.401203 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:47.401912 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:47.401919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:47.401923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:47.402047 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:47.402433 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:47.402439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:47.402443 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:47.402711 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:47.402860 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:47.403516 (+ 656us) server_negotiation.cc:299] Negotiation successful
0619 06:44:47.403725 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":254,"threads_started":1}
I20260619 06:44:47.405112 20994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:47.387617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:43505 (local address 127.16.79.65:38493)
0619 06:44:47.388329 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:47.388366 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:47.390063 (+ 1697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:47.390779 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:47.390787 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:47.391294 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:47.391897 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:47.391921 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:47.393314 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:47.393320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:47.394519 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:47.394530 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:47.395276 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:47.395282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:47.395491 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:47.396204 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:47.396226 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:47.398452 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:47.401347 (+ 2895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:47.401356 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:47.401371 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:47.401795 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:47.402152 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:47.402158 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:47.402162 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:47.402326 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:47.402852 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:47.402859 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:47.403173 (+ 314us) client_negotiation.cc:769] Sending connection context
0619 06:44:47.403428 (+ 255us) client_negotiation.cc:240] Negotiation successful
0619 06:44:47.403949 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":275,"threads_started":1}
I20260619 06:44:47.406682 20991 heartbeater.cc:347] Connected to a master server at 127.16.79.126:43505
I20260619 06:44:47.406978 20991 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:47.407629 20991 heartbeater.cc:511] Master 127.16.79.126:43505 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
I20260619 06:44:47.409832 20803 ts_manager.cc:194] Registered new tserver with Master: 4c2997a5097b4633b6ce1d620a8799f2 (127.16.79.65:39825)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
I20260619 06:44:47.411412 20803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:38493
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:44:47.431384 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43505
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:47.746398 20997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:47.746747 20997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:47.746915 20997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:47.754408 20997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:47.754527 20997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:47.754647 20997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:44:47.764472 20997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:47.765805 20997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:47.767155 20997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:47.780125 21005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:47.780192 21002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:47.780079 21003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:47.785806 20997 server_base.cc:1034] running on GCE node
I20260619 06:44:47.786517 20997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:47.787580 20997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:47.788836 20997 hybrid_clock.cc:648] HybridClock initialized: now 1781851487788784 us; error 39 us; skew 500 ppm
I20260619 06:44:47.789224 20997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:47 dist-test-slave-5w2x krb5kdc[20759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851487, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:47.792567 20997 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:44:47.793839 20997 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:47.793975 20997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:47.794312 20997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:47.797247 20997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "dc3efbaa8e07451189c040950a05b15e"
format_stamp: "Formatted at 2026-06-19 06:44:47 on dist-test-slave-5w2x"
server_key: "9e9b45c7810904596296c891309676d5"
server_key_iv: "1fb8c193d2913f3edf581266398835f0"
server_key_version: "encryptionkey@0"
I20260619 06:44:47.798074 20997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "dc3efbaa8e07451189c040950a05b15e"
format_stamp: "Formatted at 2026-06-19 06:44:47 on dist-test-slave-5w2x"
server_key: "9e9b45c7810904596296c891309676d5"
server_key_iv: "1fb8c193d2913f3edf581266398835f0"
server_key_version: "encryptionkey@0"
I20260619 06:44:47.803071 20997 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:44:47.806430 21011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:47.807684 20997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 06:44:47.807860 20997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "dc3efbaa8e07451189c040950a05b15e"
format_stamp: "Formatted at 2026-06-19 06:44:47 on dist-test-slave-5w2x"
server_key: "9e9b45c7810904596296c891309676d5"
server_key_iv: "1fb8c193d2913f3edf581266398835f0"
server_key_version: "encryptionkey@0"
I20260619 06:44:47.808096 20997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:47.823523 20997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:47.827566 20997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:47.827834 20997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:47.828927 20997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:47.830214 20997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:47.830343 20997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:47.830468 20997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:47.830556 20997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:47.863281 20997 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:39361
I20260619 06:44:47.863328 21123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:39361 every 8 connection(s)
I20260619 06:44:47.864114 20997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:44:47.869220 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 20997
I20260619 06:44:47.869414 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:44:47.869953 16701 external_mini_cluster.cc:1428] Setting key b4b16fedab232e7348bce2bb1abc5cff
Jun 19 06:44:47 dist-test-slave-5w2x krb5kdc[20759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851487, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:47.886536 20993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:47.866555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:51385 (local address 127.16.79.126:43505)
0619 06:44:47.866755 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:47.866760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:47.867847 (+ 1087us) server_negotiation.cc:407] Connection header received
0619 06:44:47.870174 (+ 2327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:47.870178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:47.870258 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:47.870346 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:47.872197 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:47.872863 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:47.875375 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:47.875760 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:47.875849 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:47.879706 (+ 3857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:47.879715 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:47.879723 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:47.879768 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:47.883306 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:47.884019 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:47.884027 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:47.884033 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:47.884208 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:47.884601 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:47.884609 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:47.884615 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:47.885019 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:47.885164 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:47.885817 (+ 653us) server_negotiation.cc:299] Negotiation successful
0619 06:44:47.886126 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 06:44:47.887427 21126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:47.867009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:43505 (local address 127.16.79.66:51385)
0619 06:44:47.867690 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:47.867726 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:47.869809 (+ 2083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:47.870606 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:47.870615 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:47.871251 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:47.871961 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:47.872003 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:47.873024 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:47.873034 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:47.875245 (+ 2211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:47.875262 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:47.875978 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:47.875987 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:47.876343 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:47.877090 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:47.877125 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:47.879457 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:47.883457 (+ 4000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:47.883463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:47.883477 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:47.883888 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:47.884320 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:47.884323 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:47.884325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:47.884489 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:47.885146 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:47.885152 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:47.885458 (+ 306us) client_negotiation.cc:769] Sending connection context
0619 06:44:47.885683 (+ 225us) client_negotiation.cc:240] Negotiation successful
0619 06:44:47.886231 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":272,"threads_started":1}
I20260619 06:44:47.889191 21124 heartbeater.cc:347] Connected to a master server at 127.16.79.126:43505
I20260619 06:44:47.889494 21124 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:47.890162 21124 heartbeater.cc:511] Master 127.16.79.126:43505 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
I20260619 06:44:47.891603 20803 ts_manager.cc:194] Registered new tserver with Master: dc3efbaa8e07451189c040950a05b15e (127.16.79.66:39361)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
I20260619 06:44:47.892560 20803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:51385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:44:47.915382 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43505
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:48.230118 21129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:48.230402 21129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:48.230543 21129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:48.237937 21129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:48.238044 21129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:48.238234 21129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:44:48.247809 21129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43505
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:48.249130 21129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:48.250463 21129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:48.262737 21134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:48.262784 21135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:48.262784 21137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:48.263639 21129 server_base.cc:1034] running on GCE node
I20260619 06:44:48.264269 21129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:48.265121 21129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:48.266309 21129 hybrid_clock.cc:648] HybridClock initialized: now 1781851488266274 us; error 29 us; skew 500 ppm
I20260619 06:44:48.266624 21129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:48 dist-test-slave-5w2x krb5kdc[20759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851488, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:48.282769 21129 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:44:48.284015 21129 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:48.284130 21129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:48.284492 21129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:48.287352 21129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "78f09cad91844ae2b5ca2105c5a3018b"
format_stamp: "Formatted at 2026-06-19 06:44:48 on dist-test-slave-5w2x"
server_key: "c1d614d218f1074d339f5be5ec2fd468"
server_key_iv: "72ffcc861ab1d1780ee7203aea633811"
server_key_version: "encryptionkey@0"
I20260619 06:44:48.288158 21129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "78f09cad91844ae2b5ca2105c5a3018b"
format_stamp: "Formatted at 2026-06-19 06:44:48 on dist-test-slave-5w2x"
server_key: "c1d614d218f1074d339f5be5ec2fd468"
server_key_iv: "72ffcc861ab1d1780ee7203aea633811"
server_key_version: "encryptionkey@0"
I20260619 06:44:48.293000 21129 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260619 06:44:48.296419 21143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:48.297701 21129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 06:44:48.297875 21129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "78f09cad91844ae2b5ca2105c5a3018b"
format_stamp: "Formatted at 2026-06-19 06:44:48 on dist-test-slave-5w2x"
server_key: "c1d614d218f1074d339f5be5ec2fd468"
server_key_iv: "72ffcc861ab1d1780ee7203aea633811"
server_key_version: "encryptionkey@0"
I20260619 06:44:48.298079 21129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:48.309125 21129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:48.313295 21129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:48.313578 21129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:48.314596 21129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:48.315949 21129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:48.316037 21129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:48.316213 21129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:48.316286 21129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:48.350919 21129 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:36505
I20260619 06:44:48.350951 21255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:36505 every 8 connection(s)
I20260619 06:44:48.351821 21129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:44:48.353775 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 21129
I20260619 06:44:48.353981 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:44:48.354547 16701 external_mini_cluster.cc:1428] Setting key ebfc3ef832db2d6719b571cfc605fe42
Jun 19 06:44:48 dist-test-slave-5w2x krb5kdc[20759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851488, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:48.371724 20993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:48.354491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:37721 (local address 127.16.79.126:43505)
0619 06:44:48.354691 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:48.354697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:48.356182 (+ 1485us) server_negotiation.cc:407] Connection header received
0619 06:44:48.358082 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:48.358085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:48.358172 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:48.358251 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:48.359703 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.360488 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.362048 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.362422 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.362508 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:48.366017 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:48.366021 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:48.366024 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:48.366055 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:48.368613 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:48.369588 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:48.369593 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:48.369596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:48.369729 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:48.370253 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:48.370257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:48.370259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:48.370513 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:48.370634 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:48.371319 (+ 685us) server_negotiation.cc:299] Negotiation successful
0619 06:44:48.371474 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:44:48.372790 21258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:48.355171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:43505 (local address 127.16.79.67:37721)
0619 06:44:48.355982 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:48.356035 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:48.357838 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:48.358469 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:48.358477 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:48.358978 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:48.359542 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:48.359559 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.360666 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.360671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:48.361923 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:48.361936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.362605 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.362610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:48.362795 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:48.363458 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:48.363480 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:48.365812 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:48.368773 (+ 2961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:48.368784 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:48.368802 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:48.369454 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:48.369884 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:48.369888 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:48.369892 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:48.370140 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:48.370642 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:48.370647 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:48.371005 (+ 358us) client_negotiation.cc:769] Sending connection context
0619 06:44:48.371235 (+ 230us) client_negotiation.cc:240] Negotiation successful
0619 06:44:48.371611 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":354,"threads_started":1}
I20260619 06:44:48.374155 21256 heartbeater.cc:347] Connected to a master server at 127.16.79.126:43505
I20260619 06:44:48.374449 21256 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:48.375161 21256 heartbeater.cc:511] Master 127.16.79.126:43505 requested a full tablet report, sending...
I20260619 06:44:48.376710 20803 ts_manager.cc:194] Registered new tserver with Master: 78f09cad91844ae2b5ca2105c5a3018b (127.16.79.67:36505)
I20260619 06:44:48.377508 20803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:37721
I20260619 06:44:48.382227 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:44:48.399539 20993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:48.387151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37982 (local address 127.16.79.126:43505)
0619 06:44:48.387342 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:48.387351 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:48.387772 (+ 421us) server_negotiation.cc:407] Connection header received
0619 06:44:48.387969 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:48.387975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:48.388050 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:48.388186 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:48.389553 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.390266 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.391534 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.391964 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.392095 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:48.394253 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:48.394259 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:48.394263 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:48.394303 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:48.396965 (+ 2662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:48.397558 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:48.397565 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:48.397569 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:48.397712 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:48.398104 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:48.398112 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:48.398116 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:48.398428 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:48.398687 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:48.399004 (+ 317us) server_negotiation.cc:299] Negotiation successful
0619 06:44:48.399211 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:44:48.403633 20803 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:44:48.410486 20803 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:44:48.414176 20991 heartbeater.cc:503] Master 127.16.79.126:43505 was elected leader, sending a full tablet report...
I20260619 06:44:48.444070 21269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:48.422818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44912 (local address 127.16.79.66:39361)
0619 06:44:48.434543 (+ 11725us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:48.434553 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:48.435659 (+ 1106us) server_negotiation.cc:407] Connection header received
0619 06:44:48.435780 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:48.435789 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:48.436083 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:48.436410 (+ 327us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:48.437995 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.439443 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.441656 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.442866 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.443019 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:48.443222 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:48.443573 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 06:44:48.443734 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11528,"thread_start_us":11386,"threads_started":1}
I20260619 06:44:48.444550 21270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:48.422580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:39361 (local address 127.0.0.1:44912)
0619 06:44:48.434659 (+ 12079us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:48.434708 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:48.434952 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:48.436767 (+ 1815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:48.436775 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:48.436828 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:48.437449 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:48.437463 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.439591 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.439596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:48.441439 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:48.441454 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.443091 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.443097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:48.443300 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:48.443364 (+ 64us) client_negotiation.cc:769] Sending connection context
0619 06:44:48.443507 (+ 143us) client_negotiation.cc:240] Negotiation successful
0619 06:44:48.443708 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11798,"mutex_wait_us":48,"spinlock_wait_cycles":1664,"thread_start_us":12796,"threads_started":1}
I20260619 06:44:48.445674 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:48.423776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39354 (local address 127.16.79.67:36505)
0619 06:44:48.436741 (+ 12965us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:48.436749 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:48.436778 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:44:48.436885 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:48.436894 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:48.437147 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:48.437384 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:48.438358 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.439722 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.442423 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.443925 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.444214 (+ 289us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:48.444429 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:48.445180 (+ 751us) server_negotiation.cc:299] Negotiation successful
0619 06:44:48.445347 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12793,"thread_start_us":12644,"threads_started":1}
I20260619 06:44:48.445861 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:48.423608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45202 (local address 127.16.79.65:39825)
0619 06:44:48.434987 (+ 11379us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:48.434997 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:48.435870 (+ 873us) server_negotiation.cc:407] Connection header received
0619 06:44:48.436178 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:48.436186 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:48.436423 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:48.436629 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:48.437450 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.439031 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.442461 (+ 3430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.443915 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.444066 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:48.444429 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:48.445353 (+ 924us) server_negotiation.cc:299] Negotiation successful
0619 06:44:48.445532 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11222,"thread_start_us":11390,"threads_started":1}
I20260619 06:44:48.446182 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:48.423273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:36505 (local address 127.0.0.1:39354)
0619 06:44:48.435884 (+ 12611us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:48.435908 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:48.436092 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:48.437368 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:48.437373 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:48.437405 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:48.438134 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:48.438148 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.439877 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.439885 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:48.442218 (+ 2333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:48.442233 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.444609 (+ 2376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.444621 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:48.444921 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:48.444979 (+ 58us) client_negotiation.cc:769] Sending connection context
0619 06:44:48.445064 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 06:44:48.445255 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12419,"mutex_wait_us":149,"thread_start_us":11000,"threads_started":1}
I20260619 06:44:48.446480 21271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:48.423260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:39825 (local address 127.0.0.1:45202)
0619 06:44:48.435769 (+ 12509us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:48.435785 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:48.435963 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:48.436766 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:48.436775 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:48.436816 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:48.437233 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:48.437246 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.439171 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.439176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:48.442281 (+ 3105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:48.442295 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.444648 (+ 2353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.444653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:48.444910 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:48.444976 (+ 66us) client_negotiation.cc:769] Sending connection context
0619 06:44:48.445069 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 06:44:48.445219 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12370,"mutex_wait_us":192,"spinlock_wait_cycles":4480,"thread_start_us":12029,"threads_started":1}
I20260619 06:44:48.448618 21059 tablet_service.cc:1467] Processing CreateTablet for tablet c714324068664662bf00dc6411e04228 (DEFAULT_TABLE table=test-table [id=08864007576a4351b1faaa2f24a84907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:48.450135 21059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c714324068664662bf00dc6411e04228. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:48.450031 20926 tablet_service.cc:1467] Processing CreateTablet for tablet c714324068664662bf00dc6411e04228 (DEFAULT_TABLE table=test-table [id=08864007576a4351b1faaa2f24a84907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:48.450666 21191 tablet_service.cc:1467] Processing CreateTablet for tablet c714324068664662bf00dc6411e04228 (DEFAULT_TABLE table=test-table [id=08864007576a4351b1faaa2f24a84907]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:48.452000 20926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c714324068664662bf00dc6411e04228. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:48.452000 21191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c714324068664662bf00dc6411e04228. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:48.462384 21274 tablet_bootstrap.cc:492] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e: Bootstrap starting.
I20260619 06:44:48.466511 21276 tablet_bootstrap.cc:492] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2: Bootstrap starting.
I20260619 06:44:48.467108 21274 tablet_bootstrap.cc:654] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:48.467641 21275 tablet_bootstrap.cc:492] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b: Bootstrap starting.
I20260619 06:44:48.468453 21274 log.cc:824] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:48.470008 21276 tablet_bootstrap.cc:654] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:48.471169 21274 tablet_bootstrap.cc:492] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e: No bootstrap required, opened a new log
I20260619 06:44:48.471184 21275 tablet_bootstrap.cc:654] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:48.471267 21276 log.cc:824] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:48.471486 21274 ts_tablet_manager.cc:1397] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260619 06:44:48.472358 21275 log.cc:824] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:48.474596 21276 tablet_bootstrap.cc:492] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2: No bootstrap required, opened a new log
I20260619 06:44:48.474850 21276 ts_tablet_manager.cc:1397] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260619 06:44:48.475162 21275 tablet_bootstrap.cc:492] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b: No bootstrap required, opened a new log
I20260619 06:44:48.475404 21275 ts_tablet_manager.cc:1397] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 06:44:48.479501 21274 raft_consensus.cc:348] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3efbaa8e07451189c040950a05b15e" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 39361 } } peers { permanent_uuid: "4c2997a5097b4633b6ce1d620a8799f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39825 } } peers { permanent_uuid: "78f09cad91844ae2b5ca2105c5a3018b" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36505 } }
I20260619 06:44:48.479902 21274 raft_consensus.cc:374] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:48.480037 21274 raft_consensus.cc:729] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3efbaa8e07451189c040950a05b15e, State: Initialized, Role: FOLLOWER
I20260619 06:44:48.480712 21274 consensus_queue.cc:260] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3efbaa8e07451189c040950a05b15e" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 39361 } } peers { permanent_uuid: "4c2997a5097b4633b6ce1d620a8799f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39825 } } peers { permanent_uuid: "78f09cad91844ae2b5ca2105c5a3018b" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36505 } }
I20260619 06:44:48.481767 21124 heartbeater.cc:503] Master 127.16.79.126:43505 was elected leader, sending a full tablet report...
I20260619 06:44:48.484794 21274 ts_tablet_manager.cc:1428] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e: Time spent starting tablet: real 0.013s user 0.004s sys 0.008s
I20260619 06:44:48.485790 21276 raft_consensus.cc:348] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3efbaa8e07451189c040950a05b15e" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 39361 } } peers { permanent_uuid: "4c2997a5097b4633b6ce1d620a8799f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39825 } } peers { permanent_uuid: "78f09cad91844ae2b5ca2105c5a3018b" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36505 } }
I20260619 06:44:48.485787 21275 raft_consensus.cc:348] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3efbaa8e07451189c040950a05b15e" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 39361 } } peers { permanent_uuid: "4c2997a5097b4633b6ce1d620a8799f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39825 } } peers { permanent_uuid: "78f09cad91844ae2b5ca2105c5a3018b" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36505 } }
I20260619 06:44:48.486219 21275 raft_consensus.cc:374] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:48.486219 21276 raft_consensus.cc:374] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:48.486322 21276 raft_consensus.cc:729] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2997a5097b4633b6ce1d620a8799f2, State: Initialized, Role: FOLLOWER
I20260619 06:44:48.486322 21275 raft_consensus.cc:729] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78f09cad91844ae2b5ca2105c5a3018b, State: Initialized, Role: FOLLOWER
I20260619 06:44:48.486920 21275 consensus_queue.cc:260] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3efbaa8e07451189c040950a05b15e" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 39361 } } peers { permanent_uuid: "4c2997a5097b4633b6ce1d620a8799f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39825 } } peers { permanent_uuid: "78f09cad91844ae2b5ca2105c5a3018b" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36505 } }
I20260619 06:44:48.486918 21276 consensus_queue.cc:260] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3efbaa8e07451189c040950a05b15e" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 39361 } } peers { permanent_uuid: "4c2997a5097b4633b6ce1d620a8799f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39825 } } peers { permanent_uuid: "78f09cad91844ae2b5ca2105c5a3018b" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36505 } }
I20260619 06:44:48.489315 21256 heartbeater.cc:503] Master 127.16.79.126:43505 was elected leader, sending a full tablet report...
I20260619 06:44:48.489655 21275 ts_tablet_manager.cc:1428] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b: Time spent starting tablet: real 0.014s user 0.004s sys 0.010s
I20260619 06:44:48.489847 21276 ts_tablet_manager.cc:1428] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
W20260619 06:44:48.603370 21257 tablet.cc:2369] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:44:48.615970 21125 tablet.cc:2369] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:44:48.637090 20992 tablet.cc:2369] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:44:48.763878 21280 raft_consensus.cc:482] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:44:48.764195 21280 raft_consensus.cc:504] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3efbaa8e07451189c040950a05b15e" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 39361 } } peers { permanent_uuid: "4c2997a5097b4633b6ce1d620a8799f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39825 } } peers { permanent_uuid: "78f09cad91844ae2b5ca2105c5a3018b" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36505 } }
I20260619 06:44:48.765848 21280 leader_election.cc:288] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c2997a5097b4633b6ce1d620a8799f2 (127.16.79.65:39825), 78f09cad91844ae2b5ca2105c5a3018b (127.16.79.67:36505)
I20260619 06:44:48.772404 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:48.766297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:51605 (local address 127.16.79.65:39825)
0619 06:44:48.766495 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:48.766502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:48.767107 (+ 605us) server_negotiation.cc:407] Connection header received
0619 06:44:48.767299 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:48.767304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:48.767384 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:48.767451 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:48.768066 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.768854 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.770610 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.771375 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.771534 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:48.771621 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:48.771983 (+ 362us) server_negotiation.cc:299] Negotiation successful
0619 06:44:48.772113 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 06:44:48.772469 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:48.766291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:52303 (local address 127.16.79.67:36505)
0619 06:44:48.766505 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:48.766512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:48.767165 (+ 653us) server_negotiation.cc:407] Connection header received
0619 06:44:48.767355 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:48.767359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:48.767414 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:48.767527 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:44:48.768218 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.768924 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.770678 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.771447 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.771611 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:48.771681 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:48.772058 (+ 377us) server_negotiation.cc:299] Negotiation successful
0619 06:44:48.772205 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:44:48.772691 21284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:48.766309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:36505 (local address 127.16.79.66:52303)
0619 06:44:48.767024 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:48.767038 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:48.767194 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:48.767565 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:48.767570 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:48.767599 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:48.768030 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:48.768043 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.769052 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.769056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:48.770498 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:48.770510 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.771672 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.771677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:48.771830 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:48.771898 (+ 68us) client_negotiation.cc:769] Sending connection context
0619 06:44:48.772008 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 06:44:48.772164 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":44,"thread_start_us":473,"threads_started":1}
I20260619 06:44:48.772989 21283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:48.766094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:39825 (local address 127.16.79.66:51605)
0619 06:44:48.766973 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:48.766993 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:48.767155 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:48.767504 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:48.767511 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:48.767534 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:44:48.767930 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:48.767939 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.768979 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.768988 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:48.770449 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:48.770463 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.771611 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.771618 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:48.771792 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:48.771857 (+ 65us) client_negotiation.cc:769] Sending connection context
0619 06:44:48.771930 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 06:44:48.772073 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":19,"thread_start_us":439,"threads_started":1}
I20260619 06:44:48.773795 21211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c714324068664662bf00dc6411e04228" candidate_uuid: "dc3efbaa8e07451189c040950a05b15e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78f09cad91844ae2b5ca2105c5a3018b" is_pre_election: true
I20260619 06:44:48.773851 20946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c714324068664662bf00dc6411e04228" candidate_uuid: "dc3efbaa8e07451189c040950a05b15e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2997a5097b4633b6ce1d620a8799f2" is_pre_election: true
I20260619 06:44:48.774277 21211 raft_consensus.cc:2454] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc3efbaa8e07451189c040950a05b15e in term 0.
I20260619 06:44:48.774314 20946 raft_consensus.cc:2454] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc3efbaa8e07451189c040950a05b15e in term 0.
I20260619 06:44:48.774931 21013 leader_election.cc:302] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78f09cad91844ae2b5ca2105c5a3018b, dc3efbaa8e07451189c040950a05b15e; no voters:
I20260619 06:44:48.775311 21280 raft_consensus.cc:2781] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:44:48.775466 21280 raft_consensus.cc:482] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:44:48.775585 21280 raft_consensus.cc:3037] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:48.777827 21280 raft_consensus.cc:504] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3efbaa8e07451189c040950a05b15e" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 39361 } } peers { permanent_uuid: "4c2997a5097b4633b6ce1d620a8799f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39825 } } peers { permanent_uuid: "78f09cad91844ae2b5ca2105c5a3018b" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36505 } }
I20260619 06:44:48.778439 21280 leader_election.cc:288] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [CANDIDATE]: Term 1 election: Requested vote from peers 4c2997a5097b4633b6ce1d620a8799f2 (127.16.79.65:39825), 78f09cad91844ae2b5ca2105c5a3018b (127.16.79.67:36505)
I20260619 06:44:48.778844 20946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c714324068664662bf00dc6411e04228" candidate_uuid: "dc3efbaa8e07451189c040950a05b15e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c2997a5097b4633b6ce1d620a8799f2"
I20260619 06:44:48.778858 21211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c714324068664662bf00dc6411e04228" candidate_uuid: "dc3efbaa8e07451189c040950a05b15e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78f09cad91844ae2b5ca2105c5a3018b"
I20260619 06:44:48.779032 20946 raft_consensus.cc:3037] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:48.779052 21211 raft_consensus.cc:3037] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:48.781198 20946 raft_consensus.cc:2454] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc3efbaa8e07451189c040950a05b15e in term 1.
I20260619 06:44:48.781200 21211 raft_consensus.cc:2454] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc3efbaa8e07451189c040950a05b15e in term 1.
I20260619 06:44:48.781615 21014 leader_election.cc:302] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c2997a5097b4633b6ce1d620a8799f2, dc3efbaa8e07451189c040950a05b15e; no voters:
I20260619 06:44:48.781932 21280 raft_consensus.cc:2781] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:48.782323 21280 raft_consensus.cc:686] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [term 1 LEADER]: Becoming Leader. State: Replica: dc3efbaa8e07451189c040950a05b15e, State: Running, Role: LEADER
I20260619 06:44:48.782789 21280 consensus_queue.cc:237] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3efbaa8e07451189c040950a05b15e" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 39361 } } peers { permanent_uuid: "4c2997a5097b4633b6ce1d620a8799f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39825 } } peers { permanent_uuid: "78f09cad91844ae2b5ca2105c5a3018b" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36505 } }
I20260619 06:44:48.787716 20803 catalog_manager.cc:5495] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e reported cstate change: term changed from 0 to 1, leader changed from <none> to dc3efbaa8e07451189c040950a05b15e (127.16.79.66). New cstate: current_term: 1 leader_uuid: "dc3efbaa8e07451189c040950a05b15e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3efbaa8e07451189c040950a05b15e" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 39361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c2997a5097b4633b6ce1d620a8799f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78f09cad91844ae2b5ca2105c5a3018b" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36505 } health_report { overall_health: UNKNOWN } } }
I20260619 06:44:48.810683 21269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:48.805711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44928 (local address 127.16.79.66:39361)
0619 06:44:48.805914 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:48.805918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:48.806055 (+ 137us) server_negotiation.cc:407] Connection header received
0619 06:44:48.806220 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:48.806224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:48.806297 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:48.806423 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:44:48.806731 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.807461 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.808780 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.809185 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.809277 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:48.809684 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:44:48.810217 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:44:48.810338 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:48.810417 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 06:44:48.810478 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 06:44:48.818828 20946 raft_consensus.cc:1261] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2 [term 1 FOLLOWER]: Refusing update from remote peer dc3efbaa8e07451189c040950a05b15e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:48.818830 21211 raft_consensus.cc:1261] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b [term 1 FOLLOWER]: Refusing update from remote peer dc3efbaa8e07451189c040950a05b15e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:48.819775 21285 consensus_queue.cc:1035] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2997a5097b4633b6ce1d620a8799f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:48.820111 21280 consensus_queue.cc:1035] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [LEADER]: Connected to new peer: Peer: permanent_uuid: "78f09cad91844ae2b5ca2105c5a3018b" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 36505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:48.831177 21288 mvcc.cc:205] Tried to move back new op lower bound from 7298463698192674816 to 7298463698057977856. Current Snapshot: MvccSnapshot[applied={T|T < 7298463698192674816}]
I20260619 06:44:48.845669 20803 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:44:48.845873 20803 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:44:48.849296 20803 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 97af174cf1c24f84a3086d65b02a905d: Sending DeleteTablet for 3 replicas of tablet c714324068664662bf00dc6411e04228
I20260619 06:44:48.850463 21059 tablet_service.cc:1514] Processing DeleteTablet for tablet c714324068664662bf00dc6411e04228 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:48 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:44912
I20260619 06:44:48.850461 20926 tablet_service.cc:1514] Processing DeleteTablet for tablet c714324068664662bf00dc6411e04228 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:48 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:45202
I20260619 06:44:48.851414 21191 tablet_service.cc:1514] Processing DeleteTablet for tablet c714324068664662bf00dc6411e04228 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:48 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:39354
I20260619 06:44:48.852196 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 20864
I20260619 06:44:48.869735 21299 tablet_replica.cc:330] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e: stopping tablet replica
I20260619 06:44:48.870297 21298 tablet_replica.cc:330] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2: stopping tablet replica
I20260619 06:44:48.870908 21298 raft_consensus.cc:2229] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:44:48.871431 21299 raft_consensus.cc:2229] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:44:48.872064 21298 raft_consensus.cc:2258] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:48.872732 21299 raft_consensus.cc:2258] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:48.874085 21300 tablet_replica.cc:330] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b: stopping tablet replica
I20260619 06:44:48.874461 21300 raft_consensus.cc:2229] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:44:48.874888 21300 raft_consensus.cc:2258] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:48.877031 21298 ts_tablet_manager.cc:1905] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:48.878953 21300 ts_tablet_manager.cc:1905] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:48.879735 21299 ts_tablet_manager.cc:1905] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:48.884424 21298 ts_tablet_manager.cc:1918] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:44:48.884670 21298 log.cc:1196] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/c714324068664662bf00dc6411e04228
I20260619 06:44:48.885285 21298 ts_tablet_manager.cc:1939] T c714324068664662bf00dc6411e04228 P 4c2997a5097b4633b6ce1d620a8799f2: Deleting consensus metadata
I20260619 06:44:48.885670 21300 ts_tablet_manager.cc:1918] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:44:48.885839 21300 log.cc:1196] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/c714324068664662bf00dc6411e04228
I20260619 06:44:48.886340 21300 ts_tablet_manager.cc:1939] T c714324068664662bf00dc6411e04228 P 78f09cad91844ae2b5ca2105c5a3018b: Deleting consensus metadata
Jun 19 06:44:48 dist-test-slave-5w2x krb5kdc[20759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:44:48.887486 20789 catalog_manager.cc:4841] TS 4c2997a5097b4633b6ce1d620a8799f2 (127.16.79.65:39825): tablet c714324068664662bf00dc6411e04228 (table test-table [id=08864007576a4351b1faaa2f24a84907]) successfully deleted
I20260619 06:44:48.888052 20788 catalog_manager.cc:4841] TS 78f09cad91844ae2b5ca2105c5a3018b (127.16.79.67:36505): tablet c714324068664662bf00dc6411e04228 (table test-table [id=08864007576a4351b1faaa2f24a84907]) successfully deleted
I20260619 06:44:48.891505 21299 ts_tablet_manager.cc:1918] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:44:48.891746 21299 log.cc:1196] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/c714324068664662bf00dc6411e04228
I20260619 06:44:48.892423 21299 ts_tablet_manager.cc:1939] T c714324068664662bf00dc6411e04228 P dc3efbaa8e07451189c040950a05b15e: Deleting consensus metadata
I20260619 06:44:48.894140 20787 catalog_manager.cc:4841] TS dc3efbaa8e07451189c040950a05b15e (127.16.79.66:39361): tablet c714324068664662bf00dc6411e04228 (table test-table [id=08864007576a4351b1faaa2f24a84907]) successfully deleted
I20260619 06:44:48.895318 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:48.853317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45214 (local address 127.16.79.65:39825)
0619 06:44:48.853535 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:48.853542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:48.860505 (+ 6963us) server_negotiation.cc:407] Connection header received
0619 06:44:48.861555 (+ 1050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:48.861563 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:48.861665 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:48.861762 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:48.871237 (+ 9475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.872780 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.877020 (+ 4240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:48.877692 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:48.877828 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:48.888462 (+ 10634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:48.888476 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:48.888493 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:48.888533 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:48.892121 (+ 3588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:48.892967 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:48.892976 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:48.892980 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:48.893136 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:48.893509 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:48.893514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:48.893517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:48.893924 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:48.894229 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:48.894737 (+ 508us) server_negotiation.cc:299] Negotiation successful
0619 06:44:48.894939 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:44:48.896719 20986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:48.988752 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 20864
I20260619 06:44:49.006026 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 20997
Jun 19 06:44:49 dist-test-slave-5w2x krb5kdc[20759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:44:49.021402 21269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:49.006772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44940 (local address 127.16.79.66:39361)
0619 06:44:49.006955 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:49.006961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:49.007032 (+ 71us) server_negotiation.cc:407] Connection header received
0619 06:44:49.007249 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:49.007254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:49.007342 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:49.007454 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:49.008715 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:49.009492 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:49.010748 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:49.011287 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:49.011424 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:49.015548 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:49.015563 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:49.015578 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:49.015620 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:49.018567 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:49.019237 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:49.019248 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:49.019260 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:49.019390 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:49.019793 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:49.019798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:49.019800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:49.020192 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:49.020525 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:49.020810 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 06:44:49.021029 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 06:44:49.022060 21119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:49.134971 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 20997
I20260619 06:44:49.152401 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 21129
Jun 19 06:44:49 dist-test-slave-5w2x krb5kdc[20759](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:44:49.166944 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:49.153159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39368 (local address 127.16.79.67:36505)
0619 06:44:49.153331 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:49.153340 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:49.153447 (+ 107us) server_negotiation.cc:407] Connection header received
0619 06:44:49.153655 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:49.153667 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:49.153736 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:49.153840 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:49.155076 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:49.155762 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:49.157084 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:49.157514 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:49.157643 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:49.161970 (+ 4327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:49.161984 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:49.161999 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:49.162032 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:49.164640 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:49.165294 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:49.165301 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:49.165303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:49.165424 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:49.165737 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:49.165740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:49.165742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:49.166084 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:49.166282 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:49.166524 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 06:44:49.166687 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:44:49.167572 21251 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:49.255401 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 21129
I20260619 06:44:49.275218 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 20772
I20260619 06:44:49.276369 20833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:49.383224 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 20772
2026-06-19T06:44:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4283 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T06:44:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:44:49Z Disabled control of system clock
I20260619 06:44:49.449527 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:42875
--webserver_interface=127.16.79.126
--webserver_port=0
--builtin_ntp_servers=127.16.79.84:46683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:42875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 06:44:49.779709 21309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:49.779994 21309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:49.780076 21309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:49.787169 21309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:44:49.787262 21309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:49.787299 21309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:49.787356 21309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:44:49.787400 21309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:44:49.796217 21309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:42875
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:42875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:49.797394 21309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:49.799041 21309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:49.810206 21314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:49.810287 21317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:49.810310 21315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:49.811626 21309 server_base.cc:1034] running on GCE node
I20260619 06:44:49.812435 21309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:49.813735 21309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:49.814949 21309 hybrid_clock.cc:648] HybridClock initialized: now 1781851489814881 us; error 31 us; skew 500 ppm
I20260619 06:44:49.815286 21309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:44:49.817440 21309 webserver.cc:460] Webserver started at http://127.16.79.126:40139/ using document root <none> and password file <none>
I20260619 06:44:49.818105 21309 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:49.818217 21309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:49.818543 21309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:49.821341 21309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "db7a6cab5416455c82d942d74a928213"
format_stamp: "Formatted at 2026-06-19 06:44:49 on dist-test-slave-5w2x"
server_key: "866c2b743149bcacc86755ed5025225a"
server_key_iv: "7ed868336a3a987fcdd09a8b0e381e47"
server_key_version: "encryptionkey@0"
I20260619 06:44:49.822113 21309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "db7a6cab5416455c82d942d74a928213"
format_stamp: "Formatted at 2026-06-19 06:44:49 on dist-test-slave-5w2x"
server_key: "866c2b743149bcacc86755ed5025225a"
server_key_iv: "7ed868336a3a987fcdd09a8b0e381e47"
server_key_version: "encryptionkey@0"
I20260619 06:44:49.826879 21309 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260619 06:44:49.830083 21323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:49.831303 21309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 06:44:49.831480 21309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "db7a6cab5416455c82d942d74a928213"
format_stamp: "Formatted at 2026-06-19 06:44:49 on dist-test-slave-5w2x"
server_key: "866c2b743149bcacc86755ed5025225a"
server_key_iv: "7ed868336a3a987fcdd09a8b0e381e47"
server_key_version: "encryptionkey@0"
I20260619 06:44:49.831691 21309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:49.837212 21309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:49.837949 21309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:49.838238 21309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:49.855818 21309 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:42875
I20260619 06:44:49.855840 21374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:42875 every 8 connection(s)
I20260619 06:44:49.857221 21309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:44:49.860819 21375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:49.866614 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 21309
I20260619 06:44:49.866822 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:44:49.867481 16701 external_mini_cluster.cc:1428] Setting key ac46015e1b639686e24d7fc77a0f0870
I20260619 06:44:49.871870 21375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213: Bootstrap starting.
I20260619 06:44:49.876066 21375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:49.877326 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:49.869604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41698 (local address 127.16.79.126:42875)
0619 06:44:49.870701 (+ 1097us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:49.870716 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:49.870771 (+ 55us) server_negotiation.cc:407] Connection header received
0619 06:44:49.873131 (+ 2360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:49.873159 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:49.873767 (+ 608us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:49.874308 (+ 541us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:49.875329 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:49.875337 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:49.875354 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:49.875372 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:49.875448 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:49.875468 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:49.875654 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:49.875771 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 06:44:49.876052 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":308,"threads_started":1}
I20260619 06:44:49.877431 21375 log.cc:824] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:49.880818 21375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213: No bootstrap required, opened a new log
I20260619 06:44:49.888124 21375 raft_consensus.cc:348] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7a6cab5416455c82d942d74a928213" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42875 } }
I20260619 06:44:49.888510 21375 raft_consensus.cc:374] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:49.888581 21375 raft_consensus.cc:729] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db7a6cab5416455c82d942d74a928213, State: Initialized, Role: FOLLOWER
I20260619 06:44:49.889107 21375 consensus_queue.cc:260] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7a6cab5416455c82d942d74a928213" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42875 } }
I20260619 06:44:49.889355 21375 raft_consensus.cc:388] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:44:49.889537 21375 raft_consensus.cc:482] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:44:49.889731 21375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:49.891901 21375 raft_consensus.cc:504] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7a6cab5416455c82d942d74a928213" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42875 } }
I20260619 06:44:49.892419 21375 leader_election.cc:302] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db7a6cab5416455c82d942d74a928213; no voters:
I20260619 06:44:49.892983 21375 leader_election.cc:288] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:44:49.893141 21380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:49.893834 21380 raft_consensus.cc:686] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [term 1 LEADER]: Becoming Leader. State: Replica: db7a6cab5416455c82d942d74a928213, State: Running, Role: LEADER
I20260619 06:44:49.894397 21380 consensus_queue.cc:237] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7a6cab5416455c82d942d74a928213" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42875 } }
I20260619 06:44:49.895056 21375 sys_catalog.cc:564] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:44:49.897672 21381 sys_catalog.cc:455] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db7a6cab5416455c82d942d74a928213" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7a6cab5416455c82d942d74a928213" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42875 } } }
I20260619 06:44:49.897719 21382 sys_catalog.cc:455] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db7a6cab5416455c82d942d74a928213. Latest consensus state: current_term: 1 leader_uuid: "db7a6cab5416455c82d942d74a928213" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7a6cab5416455c82d942d74a928213" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42875 } } }
I20260619 06:44:49.898092 21381 sys_catalog.cc:458] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:49.898833 21382 sys_catalog.cc:458] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213 [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:49.901324 21389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:44:49.906407 21389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:44:49.914207 21389 catalog_manager.cc:1347] Generated new cluster ID: 8c4321dca0474a89adc075530d082fd6
I20260619 06:44:49.914321 21389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:44:49.934445 21389 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:44:49.934999 21389 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:44:49.940361 21389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213: Generated new TSK 0
I20260619 06:44:49.941143 21389 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:44:49.949854 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42875
--builtin_ntp_servers=127.16.79.84:46683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 06:44:50.271507 21399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:50.271845 21399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:50.272014 21399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:50.279187 21399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:50.279335 21399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:50.279522 21399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:44:50.288621 21399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:50.289943 21399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:50.291687 21399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:50.305089 21405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:50.305125 21407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:50.305230 21404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:50.306339 21399 server_base.cc:1034] running on GCE node
I20260619 06:44:50.307034 21399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:50.308054 21399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:50.309311 21399 hybrid_clock.cc:648] HybridClock initialized: now 1781851490309266 us; error 74 us; skew 500 ppm
I20260619 06:44:50.309646 21399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:44:50.311896 21399 webserver.cc:460] Webserver started at http://127.16.79.65:36239/ using document root <none> and password file <none>
I20260619 06:44:50.312703 21399 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:50.312844 21399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:50.313184 21399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:50.316128 21399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "8b9305325a9f48c28580787734b4d633"
format_stamp: "Formatted at 2026-06-19 06:44:50 on dist-test-slave-5w2x"
server_key: "84bfa87eeb9a3a6a36f0b5772550fe3f"
server_key_iv: "64be42ee168f79d6e769f77711b59376"
server_key_version: "encryptionkey@0"
I20260619 06:44:50.317054 21399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "8b9305325a9f48c28580787734b4d633"
format_stamp: "Formatted at 2026-06-19 06:44:50 on dist-test-slave-5w2x"
server_key: "84bfa87eeb9a3a6a36f0b5772550fe3f"
server_key_iv: "64be42ee168f79d6e769f77711b59376"
server_key_version: "encryptionkey@0"
I20260619 06:44:50.322402 21399 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:44:50.326138 21413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:50.327469 21399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:44:50.327656 21399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "8b9305325a9f48c28580787734b4d633"
format_stamp: "Formatted at 2026-06-19 06:44:50 on dist-test-slave-5w2x"
server_key: "84bfa87eeb9a3a6a36f0b5772550fe3f"
server_key_iv: "64be42ee168f79d6e769f77711b59376"
server_key_version: "encryptionkey@0"
I20260619 06:44:50.327880 21399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:50.333709 21399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:50.334522 21399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:50.334844 21399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:50.335757 21399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:50.337225 21399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:50.337314 21399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:50.337463 21399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:50.337543 21399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:50.370371 21399 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:36387
I20260619 06:44:50.370424 21525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:36387 every 8 connection(s)
I20260619 06:44:50.371702 21399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:44:50.378918 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 21399
I20260619 06:44:50.379142 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:44:50.379963 16701 external_mini_cluster.cc:1428] Setting key ae958254c1b010401cda9f5d0f7ad415
I20260619 06:44:50.380545 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:50.374435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:39495 (local address 127.16.79.126:42875)
0619 06:44:50.374632 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:50.374636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:50.376006 (+ 1370us) server_negotiation.cc:407] Connection header received
0619 06:44:50.378009 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:50.378014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:50.378069 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:50.378167 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:50.379829 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:50.379834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:50.379837 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:50.379853 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:50.379884 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:50.379897 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:50.380064 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:50.380280 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 06:44:50.380336 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260619 06:44:50.381824 21528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:50.374945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:42875 (local address 127.16.79.65:39495)
0619 06:44:50.375847 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:50.375889 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:50.377683 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:50.378397 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:50.378410 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:50.378564 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:50.379429 (+ 865us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:44:50.379434 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:50.379489 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:44:50.379627 (+ 138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:50.380010 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:50.380023 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:50.380031 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 06:44:50.380351 (+ 320us) client_negotiation.cc:240] Negotiation successful
0619 06:44:50.380642 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":265,"threads_started":1}
I20260619 06:44:50.383123 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42875
--builtin_ntp_servers=127.16.79.84:46683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 06:44:50.383595 21526 heartbeater.cc:347] Connected to a master server at 127.16.79.126:42875
I20260619 06:44:50.383917 21526 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:50.384653 21526 heartbeater.cc:511] Master 127.16.79.126:42875 requested a full tablet report, sending...
I20260619 06:44:50.386981 21340 ts_manager.cc:194] Registered new tserver with Master: 8b9305325a9f48c28580787734b4d633 (127.16.79.65:36387)
W20260619 06:44:50.707324 21529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:50.707612 21529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:50.707764 21529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:50.715137 21529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:50.715238 21529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:50.715427 21529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:44:50.724676 21529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:50.725991 21529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:50.727792 21529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:50.741807 21534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:50.741832 21535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:50.741786 21537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:50.743027 21529 server_base.cc:1034] running on GCE node
I20260619 06:44:50.743707 21529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:50.744735 21529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:50.745987 21529 hybrid_clock.cc:648] HybridClock initialized: now 1781851490745945 us; error 61 us; skew 500 ppm
I20260619 06:44:50.746335 21529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:44:50.748844 21529 webserver.cc:460] Webserver started at http://127.16.79.66:38681/ using document root <none> and password file <none>
I20260619 06:44:50.749569 21529 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:50.749706 21529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:50.750042 21529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:50.753037 21529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "5bcf6de46af5467e834de4a1ecf0c553"
format_stamp: "Formatted at 2026-06-19 06:44:50 on dist-test-slave-5w2x"
server_key: "80c23033c0d9ec902c696469844cfda7"
server_key_iv: "8df9fb96d6561601a7c04335a56333e8"
server_key_version: "encryptionkey@0"
I20260619 06:44:50.753868 21529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "5bcf6de46af5467e834de4a1ecf0c553"
format_stamp: "Formatted at 2026-06-19 06:44:50 on dist-test-slave-5w2x"
server_key: "80c23033c0d9ec902c696469844cfda7"
server_key_iv: "8df9fb96d6561601a7c04335a56333e8"
server_key_version: "encryptionkey@0"
I20260619 06:44:50.759298 21529 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 06:44:50.762934 21543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:50.764495 21529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:44:50.764686 21529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "5bcf6de46af5467e834de4a1ecf0c553"
format_stamp: "Formatted at 2026-06-19 06:44:50 on dist-test-slave-5w2x"
server_key: "80c23033c0d9ec902c696469844cfda7"
server_key_iv: "8df9fb96d6561601a7c04335a56333e8"
server_key_version: "encryptionkey@0"
I20260619 06:44:50.764950 21529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:50.770761 21529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:50.771615 21529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:50.771901 21529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:50.772914 21529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:50.774400 21529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:50.774533 21529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:50.774679 21529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:50.774766 21529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:50.808034 21529 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:46325
I20260619 06:44:50.808096 21655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:46325 every 8 connection(s)
I20260619 06:44:50.809424 21529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:44:50.813678 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 21529
I20260619 06:44:50.813851 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:44:50.814419 16701 external_mini_cluster.cc:1428] Setting key aae81a19eaf3c6ba06434e43ae66d78d
I20260619 06:44:50.817461 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42875
--builtin_ntp_servers=127.16.79.84:46683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 06:44:50.818444 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:50.811954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:59941 (local address 127.16.79.126:42875)
0619 06:44:50.812193 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:50.812199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:50.813820 (+ 1621us) server_negotiation.cc:407] Connection header received
0619 06:44:50.815926 (+ 2106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:50.815929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:50.815977 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:50.816063 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:50.817581 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:50.817586 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:50.817592 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:50.817611 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:50.817654 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:50.817674 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:50.817801 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:50.818007 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 06:44:50.818087 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 06:44:50.819299 21658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:50.812559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:42875 (local address 127.16.79.66:59941)
0619 06:44:50.813596 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:50.813661 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:50.815639 (+ 1978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:50.816399 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:50.816413 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:50.816542 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:50.817270 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:44:50.817273 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:50.817312 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:44:50.817395 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:50.817799 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:50.817807 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:50.817813 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 06:44:50.818046 (+ 233us) client_negotiation.cc:240] Negotiation successful
0619 06:44:50.818251 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":272,"threads_started":1}
I20260619 06:44:50.820755 21656 heartbeater.cc:347] Connected to a master server at 127.16.79.126:42875
I20260619 06:44:50.821082 21656 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:50.821810 21656 heartbeater.cc:511] Master 127.16.79.126:42875 requested a full tablet report, sending...
I20260619 06:44:50.823536 21340 ts_manager.cc:194] Registered new tserver with Master: 5bcf6de46af5467e834de4a1ecf0c553 (127.16.79.66:46325)
W20260619 06:44:51.148849 21659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:51.149129 21659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:51.149322 21659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:51.156711 21659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:51.156812 21659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:51.156996 21659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:44:51.166203 21659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:51.167807 21659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:51.169809 21659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:51.182884 21664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:51.183046 21667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:51.182854 21665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:51.183660 21659 server_base.cc:1034] running on GCE node
I20260619 06:44:51.184326 21659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:51.185261 21659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:51.186489 21659 hybrid_clock.cc:648] HybridClock initialized: now 1781851491186454 us; error 51 us; skew 500 ppm
I20260619 06:44:51.186817 21659 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:44:51.189103 21659 webserver.cc:460] Webserver started at http://127.16.79.67:41065/ using document root <none> and password file <none>
I20260619 06:44:51.189798 21659 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:51.189932 21659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:51.190269 21659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:51.193086 21659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "14bbccc4e5714ebda66610134563566d"
format_stamp: "Formatted at 2026-06-19 06:44:51 on dist-test-slave-5w2x"
server_key: "4a2c06e5066fd731ab722edf20b15cc3"
server_key_iv: "fac5f2bc6da935539535937baca80bf4"
server_key_version: "encryptionkey@0"
I20260619 06:44:51.193853 21659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "14bbccc4e5714ebda66610134563566d"
format_stamp: "Formatted at 2026-06-19 06:44:51 on dist-test-slave-5w2x"
server_key: "4a2c06e5066fd731ab722edf20b15cc3"
server_key_iv: "fac5f2bc6da935539535937baca80bf4"
server_key_version: "encryptionkey@0"
I20260619 06:44:51.199231 21659 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:44:51.202651 21673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:51.203946 21659 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:44:51.204174 21659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "14bbccc4e5714ebda66610134563566d"
format_stamp: "Formatted at 2026-06-19 06:44:51 on dist-test-slave-5w2x"
server_key: "4a2c06e5066fd731ab722edf20b15cc3"
server_key_iv: "fac5f2bc6da935539535937baca80bf4"
server_key_version: "encryptionkey@0"
I20260619 06:44:51.204420 21659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:51.210579 21659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:51.211561 21659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:51.211833 21659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:51.212908 21659 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:51.214288 21659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:51.214403 21659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:51.214524 21659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:51.214599 21659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:51.247303 21659 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:45761
I20260619 06:44:51.247349 21785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:45761 every 8 connection(s)
I20260619 06:44:51.248632 21659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:44:51.256726 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.251162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:51449 (local address 127.16.79.126:42875)
0619 06:44:51.251352 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:51.251357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:51.252915 (+ 1558us) server_negotiation.cc:407] Connection header received
0619 06:44:51.254475 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:51.254479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:51.254529 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:51.254613 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:51.255920 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:51.255925 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:51.255928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:51.255945 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:51.255997 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:51.256016 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:51.256185 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:51.256408 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 06:44:51.256476 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:44:51.257704 21788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.251701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:42875 (local address 127.16.79.67:51449)
0619 06:44:51.252763 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:51.252800 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:51.254253 (+ 1453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:51.254805 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:51.254812 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:51.254898 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:51.255626 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:44:51.255628 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:51.255672 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:44:51.255758 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:51.256186 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:51.256197 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:51.256207 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 06:44:51.256475 (+ 268us) client_negotiation.cc:240] Negotiation successful
0619 06:44:51.256731 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":268,"threads_started":1}
I20260619 06:44:51.258059 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 21659
I20260619 06:44:51.258239 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:44:51.258868 16701 external_mini_cluster.cc:1428] Setting key 60062ccf2c45fd1b815804f50a9b76e9
I20260619 06:44:51.259203 21786 heartbeater.cc:347] Connected to a master server at 127.16.79.126:42875
I20260619 06:44:51.259532 21786 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:51.260301 21786 heartbeater.cc:511] Master 127.16.79.126:42875 requested a full tablet report, sending...
I20260619 06:44:51.261869 21339 ts_manager.cc:194] Registered new tserver with Master: 14bbccc4e5714ebda66610134563566d (127.16.79.67:45761)
I20260619 06:44:51.274469 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:44:51.280979 21378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.278647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41712 (local address 127.16.79.126:42875)
0619 06:44:51.278804 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:51.278808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:51.279373 (+ 565us) server_negotiation.cc:407] Connection header received
0619 06:44:51.279503 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:51.279506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:51.279556 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:51.279635 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:51.280515 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:51.280520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:51.280523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:51.280535 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:51.280559 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:51.280569 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:51.280658 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:51.280733 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 06:44:51.280774 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:44:51.284773 21339 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:44:51.291533 21339 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:44:51.309826 21798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.303512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:45761 (local address 127.0.0.1:37216)
0619 06:44:51.305013 (+ 1501us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:51.305033 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:51.305126 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:51.307397 (+ 2271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:51.307408 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:51.307535 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:51.308539 (+ 1004us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:44:51.308543 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:51.308574 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:44:51.308669 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:51.309214 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:51.309231 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:51.309237 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 06:44:51.309421 (+ 184us) client_negotiation.cc:240] Negotiation successful
0619 06:44:51.309509 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1349,"thread_start_us":650,"threads_started":1}
I20260619 06:44:51.310508 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.304673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37216 (local address 127.16.79.67:45761)
0619 06:44:51.305339 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:51.305350 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:51.305397 (+ 47us) server_negotiation.cc:407] Connection header received
0619 06:44:51.305481 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:51.305492 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:51.305811 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:51.305996 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:51.308853 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:51.308863 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:51.308881 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:51.308900 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:51.308966 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:51.308990 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:51.309164 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:51.310081 (+ 917us) server_negotiation.cc:299] Negotiation successful
0619 06:44:51.310184 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":396,"threads_started":1}
I20260619 06:44:51.313283 21799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.303437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:36387 (local address 127.0.0.1:48590)
0619 06:44:51.304325 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:51.304366 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:51.304477 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:51.311462 (+ 6985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:51.311466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:51.311489 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:51.312248 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:44:51.312251 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:51.312267 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:44:51.312315 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:51.312710 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:51.312714 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:51.312719 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 06:44:51.312829 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 06:44:51.312879 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":1245,"threads_started":1}
I20260619 06:44:51.313958 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.303609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48590 (local address 127.16.79.65:36387)
0619 06:44:51.311039 (+ 7430us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:51.311049 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:51.311079 (+ 30us) server_negotiation.cc:407] Connection header received
0619 06:44:51.311150 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:51.311156 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:51.311311 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:51.311476 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:51.312463 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:51.312471 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:51.312484 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:51.312506 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:51.312569 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:51.312592 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:51.313629 (+ 1037us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:51.313676 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 06:44:51.313748 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7272,"threads_started":1}
I20260619 06:44:51.315467 21721 tablet_service.cc:1467] Processing CreateTablet for tablet ef088f19ce5e4c3bb2889bec02e62bda (DEFAULT_TABLE table=test-table [id=b51fa9b999c948dcb1197a543ca83f95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:51.316846 21801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.304179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:46325 (local address 127.0.0.1:59906)
0619 06:44:51.308992 (+ 4813us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:51.309009 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:51.309099 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:51.314851 (+ 5752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:51.314855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:51.314877 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:51.315588 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:44:51.315592 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:51.315607 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:44:51.315673 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:51.316117 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:51.316124 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:51.316159 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 06:44:51.316329 (+ 170us) client_negotiation.cc:240] Negotiation successful
0619 06:44:51.316385 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4699,"thread_start_us":4179,"threads_started":1}
I20260619 06:44:51.317165 21721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef088f19ce5e4c3bb2889bec02e62bda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:51.320679 21803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.305309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59906 (local address 127.16.79.66:46325)
0619 06:44:51.314460 (+ 9151us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:51.314466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:51.314489 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:44:51.314547 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:51.314552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:51.314700 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:51.314852 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:51.315878 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:51.315886 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:51.315897 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:51.315919 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:51.315969 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:51.315989 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:51.316175 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:51.317101 (+ 926us) server_negotiation.cc:299] Negotiation successful
0619 06:44:51.317205 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9017,"thread_start_us":8911,"threads_started":1}
I20260619 06:44:51.320264 21461 tablet_service.cc:1467] Processing CreateTablet for tablet ef088f19ce5e4c3bb2889bec02e62bda (DEFAULT_TABLE table=test-table [id=b51fa9b999c948dcb1197a543ca83f95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:51.321740 21461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef088f19ce5e4c3bb2889bec02e62bda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:51.323496 21591 tablet_service.cc:1467] Processing CreateTablet for tablet ef088f19ce5e4c3bb2889bec02e62bda (DEFAULT_TABLE table=test-table [id=b51fa9b999c948dcb1197a543ca83f95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:44:51.324625 21591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef088f19ce5e4c3bb2889bec02e62bda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:51.333424 21804 tablet_bootstrap.cc:492] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d: Bootstrap starting.
I20260619 06:44:51.337045 21804 tablet_bootstrap.cc:654] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:51.338184 21804 log.cc:824] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:51.343533 21806 tablet_bootstrap.cc:492] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553: Bootstrap starting.
I20260619 06:44:51.343870 21805 tablet_bootstrap.cc:492] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633: Bootstrap starting.
I20260619 06:44:51.346838 21806 tablet_bootstrap.cc:654] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:51.348347 21806 log.cc:824] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:51.350095 21805 tablet_bootstrap.cc:654] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:51.351071 21805 log.cc:824] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:51.351500 21804 tablet_bootstrap.cc:492] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d: No bootstrap required, opened a new log
I20260619 06:44:51.351500 21806 tablet_bootstrap.cc:492] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553: No bootstrap required, opened a new log
I20260619 06:44:51.351788 21804 ts_tablet_manager.cc:1397] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.012s
I20260619 06:44:51.351788 21806 ts_tablet_manager.cc:1397] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260619 06:44:51.353858 21805 tablet_bootstrap.cc:492] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633: No bootstrap required, opened a new log
I20260619 06:44:51.354136 21805 ts_tablet_manager.cc:1397] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260619 06:44:51.361725 21805 raft_consensus.cc:348] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b9305325a9f48c28580787734b4d633" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36387 } } peers { permanent_uuid: "14bbccc4e5714ebda66610134563566d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45761 } } peers { permanent_uuid: "5bcf6de46af5467e834de4a1ecf0c553" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46325 } }
I20260619 06:44:51.362129 21805 raft_consensus.cc:374] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:51.362221 21805 raft_consensus.cc:729] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b9305325a9f48c28580787734b4d633, State: Initialized, Role: FOLLOWER
I20260619 06:44:51.362859 21805 consensus_queue.cc:260] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b9305325a9f48c28580787734b4d633" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36387 } } peers { permanent_uuid: "14bbccc4e5714ebda66610134563566d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45761 } } peers { permanent_uuid: "5bcf6de46af5467e834de4a1ecf0c553" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46325 } }
I20260619 06:44:51.363967 21526 heartbeater.cc:503] Master 127.16.79.126:42875 was elected leader, sending a full tablet report...
I20260619 06:44:51.363668 21804 raft_consensus.cc:348] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b9305325a9f48c28580787734b4d633" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36387 } } peers { permanent_uuid: "14bbccc4e5714ebda66610134563566d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45761 } } peers { permanent_uuid: "5bcf6de46af5467e834de4a1ecf0c553" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46325 } }
I20260619 06:44:51.363668 21806 raft_consensus.cc:348] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b9305325a9f48c28580787734b4d633" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36387 } } peers { permanent_uuid: "14bbccc4e5714ebda66610134563566d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45761 } } peers { permanent_uuid: "5bcf6de46af5467e834de4a1ecf0c553" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46325 } }
I20260619 06:44:51.364117 21804 raft_consensus.cc:374] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:51.364184 21806 raft_consensus.cc:374] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:51.364266 21804 raft_consensus.cc:729] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14bbccc4e5714ebda66610134563566d, State: Initialized, Role: FOLLOWER
I20260619 06:44:51.364280 21806 raft_consensus.cc:729] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bcf6de46af5467e834de4a1ecf0c553, State: Initialized, Role: FOLLOWER
I20260619 06:44:51.364944 21805 ts_tablet_manager.cc:1428] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260619 06:44:51.364961 21806 consensus_queue.cc:260] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b9305325a9f48c28580787734b4d633" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36387 } } peers { permanent_uuid: "14bbccc4e5714ebda66610134563566d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45761 } } peers { permanent_uuid: "5bcf6de46af5467e834de4a1ecf0c553" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46325 } }
I20260619 06:44:51.365020 21804 consensus_queue.cc:260] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b9305325a9f48c28580787734b4d633" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36387 } } peers { permanent_uuid: "14bbccc4e5714ebda66610134563566d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45761 } } peers { permanent_uuid: "5bcf6de46af5467e834de4a1ecf0c553" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46325 } }
I20260619 06:44:51.366585 21656 heartbeater.cc:503] Master 127.16.79.126:42875 was elected leader, sending a full tablet report...
I20260619 06:44:51.366919 21804 ts_tablet_manager.cc:1428] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260619 06:44:51.367184 21806 ts_tablet_manager.cc:1428] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553: Time spent starting tablet: real 0.015s user 0.009s sys 0.008s
I20260619 06:44:51.367939 21786 heartbeater.cc:503] Master 127.16.79.126:42875 was elected leader, sending a full tablet report...
W20260619 06:44:51.373950 21527 tablet.cc:2369] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:44:51.500383 21787 tablet.cc:2369] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:44:51.520780 21811 raft_consensus.cc:482] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:44:51.521140 21811 raft_consensus.cc:504] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b9305325a9f48c28580787734b4d633" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36387 } } peers { permanent_uuid: "14bbccc4e5714ebda66610134563566d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45761 } } peers { permanent_uuid: "5bcf6de46af5467e834de4a1ecf0c553" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46325 } }
I20260619 06:44:51.523013 21811 leader_election.cc:288] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b9305325a9f48c28580787734b4d633 (127.16.79.65:36387), 14bbccc4e5714ebda66610134563566d (127.16.79.67:45761)
I20260619 06:44:51.526577 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.523470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:57047 (local address 127.16.79.67:45761)
0619 06:44:51.523658 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:51.523666 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:51.524446 (+ 780us) server_negotiation.cc:407] Connection header received
0619 06:44:51.524664 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:51.524669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:51.524729 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:51.524836 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:51.525938 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:51.525944 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:51.525948 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:51.525967 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:51.526008 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:51.526020 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:51.526150 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:51.526239 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 06:44:51.526303 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:44:51.526782 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.523602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:59911 (local address 127.16.79.65:36387)
0619 06:44:51.523773 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:51.523777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:51.524920 (+ 1143us) server_negotiation.cc:407] Connection header received
0619 06:44:51.525126 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:51.525132 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:51.525188 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:51.525290 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:51.526145 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:51.526151 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:51.526155 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:51.526172 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:51.526207 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:51.526217 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:51.526332 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:51.526439 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 06:44:51.526501 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 06:44:51.526788 21813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.523508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:45761 (local address 127.16.79.66:57047)
0619 06:44:51.524331 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:51.524350 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:51.524509 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:51.524882 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:51.524889 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:51.524915 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:51.525726 (+ 811us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:44:51.525731 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:51.525752 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:44:51.525806 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:51.526114 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:51.526118 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:51.526133 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 06:44:51.526277 (+ 144us) client_negotiation.cc:240] Negotiation successful
0619 06:44:51.526331 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":448,"threads_started":1}
I20260619 06:44:51.527011 21814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.523955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:36387 (local address 127.16.79.66:59911)
0619 06:44:51.524763 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:51.524778 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:51.524967 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:51.525310 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:51.525315 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:51.525339 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:51.525958 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:44:51.525960 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:51.525973 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:44:51.526007 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:51.526307 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:51.526311 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:51.526316 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 06:44:51.526488 (+ 172us) client_negotiation.cc:240] Negotiation successful
0619 06:44:51.526536 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":494,"threads_started":1}
I20260619 06:44:51.527822 21741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef088f19ce5e4c3bb2889bec02e62bda" candidate_uuid: "5bcf6de46af5467e834de4a1ecf0c553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14bbccc4e5714ebda66610134563566d" is_pre_election: true
I20260619 06:44:51.527858 21481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef088f19ce5e4c3bb2889bec02e62bda" candidate_uuid: "5bcf6de46af5467e834de4a1ecf0c553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b9305325a9f48c28580787734b4d633" is_pre_election: true
I20260619 06:44:51.528337 21741 raft_consensus.cc:2454] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bcf6de46af5467e834de4a1ecf0c553 in term 0.
I20260619 06:44:51.528373 21481 raft_consensus.cc:2454] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5bcf6de46af5467e834de4a1ecf0c553 in term 0.
I20260619 06:44:51.528991 21545 leader_election.cc:302] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bcf6de46af5467e834de4a1ecf0c553, 8b9305325a9f48c28580787734b4d633; no voters:
I20260619 06:44:51.529392 21811 raft_consensus.cc:2781] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:44:51.529541 21811 raft_consensus.cc:482] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:44:51.529644 21811 raft_consensus.cc:3037] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:51.531806 21811 raft_consensus.cc:504] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b9305325a9f48c28580787734b4d633" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36387 } } peers { permanent_uuid: "14bbccc4e5714ebda66610134563566d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45761 } } peers { permanent_uuid: "5bcf6de46af5467e834de4a1ecf0c553" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46325 } }
I20260619 06:44:51.532575 21811 leader_election.cc:288] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [CANDIDATE]: Term 1 election: Requested vote from peers 8b9305325a9f48c28580787734b4d633 (127.16.79.65:36387), 14bbccc4e5714ebda66610134563566d (127.16.79.67:45761)
I20260619 06:44:51.532932 21481 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef088f19ce5e4c3bb2889bec02e62bda" candidate_uuid: "5bcf6de46af5467e834de4a1ecf0c553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b9305325a9f48c28580787734b4d633"
I20260619 06:44:51.533066 21741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef088f19ce5e4c3bb2889bec02e62bda" candidate_uuid: "5bcf6de46af5467e834de4a1ecf0c553" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14bbccc4e5714ebda66610134563566d"
I20260619 06:44:51.533125 21481 raft_consensus.cc:3037] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:51.533293 21741 raft_consensus.cc:3037] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:51.535447 21741 raft_consensus.cc:2454] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bcf6de46af5467e834de4a1ecf0c553 in term 1.
I20260619 06:44:51.535604 21481 raft_consensus.cc:2454] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bcf6de46af5467e834de4a1ecf0c553 in term 1.
I20260619 06:44:51.535874 21547 leader_election.cc:302] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14bbccc4e5714ebda66610134563566d, 5bcf6de46af5467e834de4a1ecf0c553; no voters:
I20260619 06:44:51.536237 21811 raft_consensus.cc:2781] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:51.536710 21811 raft_consensus.cc:686] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [term 1 LEADER]: Becoming Leader. State: Replica: 5bcf6de46af5467e834de4a1ecf0c553, State: Running, Role: LEADER
I20260619 06:44:51.537232 21811 consensus_queue.cc:237] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b9305325a9f48c28580787734b4d633" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36387 } } peers { permanent_uuid: "14bbccc4e5714ebda66610134563566d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45761 } } peers { permanent_uuid: "5bcf6de46af5467e834de4a1ecf0c553" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46325 } }
I20260619 06:44:51.541741 21339 catalog_manager.cc:5495] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5bcf6de46af5467e834de4a1ecf0c553 (127.16.79.66). New cstate: current_term: 1 leader_uuid: "5bcf6de46af5467e834de4a1ecf0c553" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b9305325a9f48c28580787734b4d633" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14bbccc4e5714ebda66610134563566d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5bcf6de46af5467e834de4a1ecf0c553" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 46325 } health_report { overall_health: HEALTHY } } }
I20260619 06:44:51.555761 21803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.554033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59916 (local address 127.16.79.66:46325)
0619 06:44:51.554201 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:51.554205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:51.554222 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:44:51.554317 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:51.554320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:51.554391 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:51.554469 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:51.555328 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:51.555332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:51.555334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:51.555346 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:51.555370 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:51.555380 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:51.555469 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:51.555534 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 06:44:51.555584 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260619 06:44:51.561367 21657 tablet.cc:2369] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:44:51.563934 21741 raft_consensus.cc:1261] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d [term 1 FOLLOWER]: Refusing update from remote peer 5bcf6de46af5467e834de4a1ecf0c553: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:51.563948 21481 raft_consensus.cc:1261] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633 [term 1 FOLLOWER]: Refusing update from remote peer 5bcf6de46af5467e834de4a1ecf0c553: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:44:51.564945 21815 consensus_queue.cc:1035] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b9305325a9f48c28580787734b4d633" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:51.565275 21811 consensus_queue.cc:1035] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14bbccc4e5714ebda66610134563566d" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:44:51.576973 21819 mvcc.cc:205] Tried to move back new op lower bound from 7298463709435781120 to 7298463709340237824. Current Snapshot: MvccSnapshot[applied={T|T < 7298463709435781120}]
I20260619 06:44:51.588235 21820 mvcc.cc:205] Tried to move back new op lower bound from 7298463709435781120 to 7298463709340237824. Current Snapshot: MvccSnapshot[applied={T|T < 7298463709435781120 or (T in {7298463709435781120})}]
I20260619 06:44:51.591572 21339 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:44:51.591895 21339 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:44:51.595465 21339 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db7a6cab5416455c82d942d74a928213: Sending DeleteTablet for 3 replicas of tablet ef088f19ce5e4c3bb2889bec02e62bda
I20260619 06:44:51.596858 21461 tablet_service.cc:1514] Processing DeleteTablet for tablet ef088f19ce5e4c3bb2889bec02e62bda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:51 UTC) from {username='slave'} at 127.0.0.1:48590
I20260619 06:44:51.597005 21721 tablet_service.cc:1514] Processing DeleteTablet for tablet ef088f19ce5e4c3bb2889bec02e62bda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:51 UTC) from {username='slave'} at 127.0.0.1:37216
I20260619 06:44:51.597011 21591 tablet_service.cc:1514] Processing DeleteTablet for tablet ef088f19ce5e4c3bb2889bec02e62bda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:44:51 UTC) from {username='slave'} at 127.0.0.1:59906
I20260619 06:44:51.599150 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 21399
I20260619 06:44:51.612159 21828 tablet_replica.cc:330] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633: stopping tablet replica
I20260619 06:44:51.612651 21828 raft_consensus.cc:2229] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:44:51.613183 21828 raft_consensus.cc:2258] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:51.615533 21828 ts_tablet_manager.cc:1905] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:51.622437 21828 ts_tablet_manager.cc:1918] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:44:51.622611 21828 log.cc:1196] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/ef088f19ce5e4c3bb2889bec02e62bda
I20260619 06:44:51.623117 21828 ts_tablet_manager.cc:1939] T ef088f19ce5e4c3bb2889bec02e62bda P 8b9305325a9f48c28580787734b4d633: Deleting consensus metadata
I20260619 06:44:51.625182 21325 catalog_manager.cc:4841] TS 8b9305325a9f48c28580787734b4d633 (127.16.79.65:36387): tablet ef088f19ce5e4c3bb2889bec02e62bda (table test-table [id=b51fa9b999c948dcb1197a543ca83f95]) successfully deleted
I20260619 06:44:51.627116 21829 tablet_replica.cc:330] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d: stopping tablet replica
I20260619 06:44:51.627792 21830 tablet_replica.cc:330] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553: stopping tablet replica
I20260619 06:44:51.627959 21800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.601032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.16.79.65:36387)
0619 06:44:51.601281 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:51.601290 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:51.625102 (+ 23812us) server_negotiation.cc:407] Connection header received
0619 06:44:51.625794 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:51.625800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:51.625897 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:51.626016 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:51.627192 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:51.627197 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:51.627202 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:51.627222 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:51.627256 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:51.627271 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:51.627364 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:51.627624 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 06:44:51.627687 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 06:44:51.628470 21829 raft_consensus.cc:2229] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:44:51.628590 21830 raft_consensus.cc:2229] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:44:51.628886 21521 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:51.629163 21829 raft_consensus.cc:2258] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:51.629263 21830 raft_consensus.cc:2258] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:44:51.631620 21829 ts_tablet_manager.cc:1905] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:51.631594 21830 ts_tablet_manager.cc:1905] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:44:51.638696 21829 ts_tablet_manager.cc:1918] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:44:51.638936 21829 log.cc:1196] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/ef088f19ce5e4c3bb2889bec02e62bda
I20260619 06:44:51.639119 21830 ts_tablet_manager.cc:1918] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:44:51.639289 21830 log.cc:1196] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/ef088f19ce5e4c3bb2889bec02e62bda
I20260619 06:44:51.639668 21829 ts_tablet_manager.cc:1939] T ef088f19ce5e4c3bb2889bec02e62bda P 14bbccc4e5714ebda66610134563566d: Deleting consensus metadata
I20260619 06:44:51.639820 21830 ts_tablet_manager.cc:1939] T ef088f19ce5e4c3bb2889bec02e62bda P 5bcf6de46af5467e834de4a1ecf0c553: Deleting consensus metadata
I20260619 06:44:51.641673 21327 catalog_manager.cc:4841] TS 14bbccc4e5714ebda66610134563566d (127.16.79.67:45761): tablet ef088f19ce5e4c3bb2889bec02e62bda (table test-table [id=b51fa9b999c948dcb1197a543ca83f95]) successfully deleted
I20260619 06:44:51.641700 21326 catalog_manager.cc:4841] TS 5bcf6de46af5467e834de4a1ecf0c553 (127.16.79.66:46325): tablet ef088f19ce5e4c3bb2889bec02e62bda (table test-table [id=b51fa9b999c948dcb1197a543ca83f95]) successfully deleted
I20260619 06:44:51.727447 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 21399
I20260619 06:44:51.749764 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 21529
I20260619 06:44:51.753047 21803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.750550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59920 (local address 127.16.79.66:46325)
0619 06:44:51.750730 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:51.750736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:51.750874 (+ 138us) server_negotiation.cc:407] Connection header received
0619 06:44:51.751117 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:51.751121 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:51.751169 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:51.751283 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:51.752355 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:51.752362 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:51.752365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:51.752380 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:51.752409 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:51.752421 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:51.752589 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:51.752695 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 06:44:51.752750 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:44:51.753724 21651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:51.868184 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 21529
I20260619 06:44:51.885439 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 21659
I20260619 06:44:51.887848 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:51.886116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37226 (local address 127.16.79.67:45761)
0619 06:44:51.886267 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:51.886271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:51.886288 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:44:51.886443 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:51.886447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:51.886515 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:51.886602 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:51.887380 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:51.887385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:51.887388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:51.887402 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:51.887432 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:51.887443 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:51.887548 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:51.887602 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 06:44:51.887644 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 06:44:51.888468 21781 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:51.976032 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 21659
I20260619 06:44:51.993475 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 21309
I20260619 06:44:51.994747 21370 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:52.099606 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 21309
2026-06-19T06:44:52Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2714 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 06:44:52.139793 16701 mini_oidc.cc:152] Starting JWKS server
I20260619 06:44:52.142333 16701 webserver.cc:460] Webserver started at https://127.0.0.1:39329/ using document root <none> and password file <none>
I20260619 06:44:52.142652 16701 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 06:44:52.143261 16701 webserver.cc:460] Webserver started at http://127.0.0.1:38057/ using document root <none> and password file <none>
2026-06-19T06:44:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:44:52Z Disabled control of system clock
I20260619 06:44:52.167968 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:39147
--webserver_interface=127.16.79.126
--webserver_port=0
--builtin_ntp_servers=127.16.79.84:36447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:39147
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39329/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/testchainca.cert with env {}
W20260619 06:44:52.487406 21841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:52.487753 21841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:52.487905 21841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:52.495215 21841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:44:52.495359 21841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:52.495448 21841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:52.495538 21841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:44:52.495621 21841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 06:44:52.495708 21841 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 06:44:52.495786 21841 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39329/jwks.json
I20260619 06:44:52.505697 21841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:36447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:39147
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:39147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39329/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:52.506970 21841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:52.508342 21841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:52.519909 21846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:52.519960 21849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:52.519996 21847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:52.520828 21841 server_base.cc:1034] running on GCE node
I20260619 06:44:52.521606 21841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:52.522970 21841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:52.524235 21841 hybrid_clock.cc:648] HybridClock initialized: now 1781851492524177 us; error 51 us; skew 500 ppm
I20260619 06:44:52.524580 21841 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:44:52.526822 21841 webserver.cc:460] Webserver started at http://127.16.79.126:34637/ using document root <none> and password file <none>
I20260619 06:44:52.527513 21841 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:52.527659 21841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:52.527966 21841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:52.530962 21841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "d099382f95834c3db4b2559bec317b1b"
format_stamp: "Formatted at 2026-06-19 06:44:52 on dist-test-slave-5w2x"
server_key: "40112e624ffab519e5bbf5ed9af57dee"
server_key_iv: "cddf8265cdfe8f05b361b017b7dbf91c"
server_key_version: "encryptionkey@0"
I20260619 06:44:52.531762 21841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "d099382f95834c3db4b2559bec317b1b"
format_stamp: "Formatted at 2026-06-19 06:44:52 on dist-test-slave-5w2x"
server_key: "40112e624ffab519e5bbf5ed9af57dee"
server_key_iv: "cddf8265cdfe8f05b361b017b7dbf91c"
server_key_version: "encryptionkey@0"
I20260619 06:44:52.536755 21841 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:44:52.540110 21855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:52.541435 21841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:44:52.541635 21841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "d099382f95834c3db4b2559bec317b1b"
format_stamp: "Formatted at 2026-06-19 06:44:52 on dist-test-slave-5w2x"
server_key: "40112e624ffab519e5bbf5ed9af57dee"
server_key_iv: "cddf8265cdfe8f05b361b017b7dbf91c"
server_key_version: "encryptionkey@0"
I20260619 06:44:52.541857 21841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:52.556195 21841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:52.557085 21841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:52.557403 21841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:52.575600 21841 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:39147
I20260619 06:44:52.575623 21906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:39147 every 8 connection(s)
I20260619 06:44:52.577008 21841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:44:52.581174 21907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:52.585551 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 21841
I20260619 06:44:52.585777 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:44:52.586524 16701 external_mini_cluster.cc:1428] Setting key 6a3b044865d09f33cf91dfc7b0df57c4
I20260619 06:44:52.593227 21907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b: Bootstrap starting.
I20260619 06:44:52.597003 21907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:52.598320 21907 log.cc:824] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:52.601250 21910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:52.588545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50946 (local address 127.16.79.126:39147)
0619 06:44:52.589548 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:52.589559 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:52.589603 (+ 44us) server_negotiation.cc:407] Connection header received
0619 06:44:52.591766 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:52.591789 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:52.592331 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:52.592685 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:52.593311 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:52.594678 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:52.596587 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:52.597304 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:52.597463 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:52.598674 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:52.598682 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:52.598696 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:52.599481 (+ 785us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:52.599598 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:52.599626 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:52.599754 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:52.599962 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 06:44:52.600334 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":361,"threads_started":1}
I20260619 06:44:52.601408 21907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b: No bootstrap required, opened a new log
I20260619 06:44:52.605166 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:44:52.609879 21907 raft_consensus.cc:348] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099382f95834c3db4b2559bec317b1b" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39147 } }
I20260619 06:44:52.610251 21907 raft_consensus.cc:374] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:52.610373 21907 raft_consensus.cc:729] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d099382f95834c3db4b2559bec317b1b, State: Initialized, Role: FOLLOWER
I20260619 06:44:52.610925 21907 consensus_queue.cc:260] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099382f95834c3db4b2559bec317b1b" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39147 } }
I20260619 06:44:52.611150 21907 raft_consensus.cc:388] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:44:52.611281 21907 raft_consensus.cc:482] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:44:52.611449 21907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:52.614146 21907 raft_consensus.cc:504] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099382f95834c3db4b2559bec317b1b" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39147 } }
I20260619 06:44:52.614643 21907 leader_election.cc:302] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d099382f95834c3db4b2559bec317b1b; no voters:
I20260619 06:44:52.615206 21907 leader_election.cc:288] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:44:52.615378 21912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:52.616230 21912 raft_consensus.cc:686] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [term 1 LEADER]: Becoming Leader. State: Replica: d099382f95834c3db4b2559bec317b1b, State: Running, Role: LEADER
I20260619 06:44:52.616827 21912 consensus_queue.cc:237] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099382f95834c3db4b2559bec317b1b" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39147 } }
I20260619 06:44:52.617377 21907 sys_catalog.cc:564] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:44:52.619707 21913 sys_catalog.cc:455] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d099382f95834c3db4b2559bec317b1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099382f95834c3db4b2559bec317b1b" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39147 } } }
I20260619 06:44:52.620718 21913 sys_catalog.cc:458] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:52.620908 21914 sys_catalog.cc:455] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d099382f95834c3db4b2559bec317b1b. Latest consensus state: current_term: 1 leader_uuid: "d099382f95834c3db4b2559bec317b1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d099382f95834c3db4b2559bec317b1b" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39147 } } }
I20260619 06:44:52.621357 21914 sys_catalog.cc:458] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:52.627542 21920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:44:52.631687 21920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:44:52.639967 21920 catalog_manager.cc:1347] Generated new cluster ID: 56d64ef91e424a3e96897a314e11aa17
I20260619 06:44:52.640081 21920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:44:52.667176 21920 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:44:52.668991 21920 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:44:52.677399 21920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d099382f95834c3db4b2559bec317b1b: Generated new TSK 0
I20260619 06:44:52.678330 21920 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 06:44:52.726325 21937 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 06:44:52.726502 21937 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 06:44:52.727480 21910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:52.706153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50950 (local address 127.16.79.126:39147)
0619 06:44:52.706383 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:52.706392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:52.706963 (+ 571us) server_negotiation.cc:407] Connection header received
0619 06:44:52.707288 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:52.707292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:52.707361 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:52.707483 (+ 122us) server_negotiation.cc:226] Negotiated authn=JWT
0619 06:44:52.708103 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:52.708884 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:52.710972 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:52.711381 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:52.711468 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:52.712056 (+ 588us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 06:44:52.726929 (+ 14873us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39329/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 06:44:52.727263 (+ 334us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50950: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39329/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":72}
W20260619 06:44:52.727641 21910 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50950: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39329/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 06:44:52.729856 21936 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:44:52.706149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:39147 (local address 127.0.0.1:50950)
0619 06:44:52.706853 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:52.706875 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:52.707063 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:52.707540 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:52.707546 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:52.707559 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0619 06:44:52.707954 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:52.707965 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:52.709016 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:52.709022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:52.710831 (+ 1809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:52.710848 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:52.711542 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:52.711548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:52.711734 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:52.711816 (+ 82us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 06:44:52.727230 (+ 15414us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39329/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 06:44:52.727619 (+ 389us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.79.126:39147: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39329/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":351,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.16.79.126:39147: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39329/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 06:44:52.731819 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 21841
I20260619 06:44:52.733011 21902 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:52.825700 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 21841
2026-06-19T06:44:52Z chronyd exiting
I20260619 06:44:52.875303 16701 test_util.cc:175] -----------------------------------------------
I20260619 06:44:52.875442 16701 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781851451399607-16701-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (739 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 06:44:52.879016 16701 mini_oidc.cc:152] Starting JWKS server
I20260619 06:44:52.880862 16701 webserver.cc:460] Webserver started at https://127.0.0.1:44855/ using document root <none> and password file <none>
I20260619 06:44:52.881042 16701 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 06:44:52.881630 16701 webserver.cc:460] Webserver started at http://127.0.0.1:46543/ using document root <none> and password file <none>
2026-06-19T06:44:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:44:52Z Disabled control of system clock
I20260619 06:44:52.908277 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:46029
--webserver_interface=127.16.79.126
--webserver_port=0
--builtin_ntp_servers=127.16.79.84:34747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:46029
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44855/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/testchainca.cert with env {}
W20260619 06:44:53.231284 21944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:53.231586 21944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:53.231742 21944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:53.239159 21944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:44:53.239280 21944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:53.239391 21944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:53.239490 21944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:44:53.239553 21944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 06:44:53.239634 21944 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 06:44:53.239709 21944 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44855/jwks.json
I20260619 06:44:53.250057 21944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:34747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:46029
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:46029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44855/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:53.251395 21944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:53.252701 21944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:53.265201 21949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:53.265695 21952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:53.265890 21950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:53.266523 21944 server_base.cc:1034] running on GCE node
I20260619 06:44:53.267398 21944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:53.269418 21944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:53.270684 21944 hybrid_clock.cc:648] HybridClock initialized: now 1781851493270633 us; error 36 us; skew 500 ppm
I20260619 06:44:53.271059 21944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:44:53.273330 21944 webserver.cc:460] Webserver started at http://127.16.79.126:43593/ using document root <none> and password file <none>
I20260619 06:44:53.274015 21944 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:53.274150 21944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:53.274468 21944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:53.277331 21944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "01fb65ef4264498b8707db1ff9987e2e"
format_stamp: "Formatted at 2026-06-19 06:44:53 on dist-test-slave-5w2x"
server_key: "094946a90c6388b2be26d7764a920a04"
server_key_iv: "489454384bffabed596a23c904f29561"
server_key_version: "encryptionkey@0"
I20260619 06:44:53.278115 21944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "01fb65ef4264498b8707db1ff9987e2e"
format_stamp: "Formatted at 2026-06-19 06:44:53 on dist-test-slave-5w2x"
server_key: "094946a90c6388b2be26d7764a920a04"
server_key_iv: "489454384bffabed596a23c904f29561"
server_key_version: "encryptionkey@0"
I20260619 06:44:53.283034 21944 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:44:53.286329 21958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:53.287563 21944 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260619 06:44:53.287742 21944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "01fb65ef4264498b8707db1ff9987e2e"
format_stamp: "Formatted at 2026-06-19 06:44:53 on dist-test-slave-5w2x"
server_key: "094946a90c6388b2be26d7764a920a04"
server_key_iv: "489454384bffabed596a23c904f29561"
server_key_version: "encryptionkey@0"
I20260619 06:44:53.287951 21944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:53.306903 21944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:53.307757 21944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:53.308076 21944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:53.326162 21944 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:46029
I20260619 06:44:53.326186 22009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:46029 every 8 connection(s)
I20260619 06:44:53.327498 21944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:44:53.331111 22010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:53.336074 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 21944
I20260619 06:44:53.336306 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:44:53.336926 16701 external_mini_cluster.cc:1428] Setting key 23636c832649a298940cfd5c60b8202e
I20260619 06:44:53.343627 22010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e: Bootstrap starting.
I20260619 06:44:53.347221 22010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:53.348559 22010 log.cc:824] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:53.351778 22010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e: No bootstrap required, opened a new log
I20260619 06:44:53.352039 22013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:53.338561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58108 (local address 127.16.79.126:46029)
0619 06:44:53.339635 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:53.339649 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:53.339702 (+ 53us) server_negotiation.cc:407] Connection header received
0619 06:44:53.342145 (+ 2443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:53.342173 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:53.342760 (+ 587us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:53.343186 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:53.343730 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:53.345711 (+ 1981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:53.346986 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:53.348062 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:53.348301 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:53.349431 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:53.349438 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:53.349451 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:53.350066 (+ 615us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:53.350161 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:53.350186 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:53.350288 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:53.350501 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 06:44:53.350812 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":346,"threads_started":1}
I20260619 06:44:53.355773 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:44:53.360056 22010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01fb65ef4264498b8707db1ff9987e2e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46029 } }
I20260619 06:44:53.360530 22010 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:53.360641 22010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01fb65ef4264498b8707db1ff9987e2e, State: Initialized, Role: FOLLOWER
I20260619 06:44:53.361171 22010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01fb65ef4264498b8707db1ff9987e2e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46029 } }
I20260619 06:44:53.361399 22010 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:44:53.361542 22010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:44:53.361716 22010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:53.364013 22010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01fb65ef4264498b8707db1ff9987e2e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46029 } }
I20260619 06:44:53.364593 22010 leader_election.cc:302] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01fb65ef4264498b8707db1ff9987e2e; no voters:
I20260619 06:44:53.365152 22010 leader_election.cc:288] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:44:53.365320 22015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:53.366456 22015 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [term 1 LEADER]: Becoming Leader. State: Replica: 01fb65ef4264498b8707db1ff9987e2e, State: Running, Role: LEADER
I20260619 06:44:53.366997 22015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01fb65ef4264498b8707db1ff9987e2e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46029 } }
I20260619 06:44:53.367246 22010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:44:53.370100 22017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01fb65ef4264498b8707db1ff9987e2e. Latest consensus state: current_term: 1 leader_uuid: "01fb65ef4264498b8707db1ff9987e2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01fb65ef4264498b8707db1ff9987e2e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46029 } } }
I20260619 06:44:53.370474 22017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:53.370203 22016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01fb65ef4264498b8707db1ff9987e2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01fb65ef4264498b8707db1ff9987e2e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46029 } } }
I20260619 06:44:53.370620 22016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:53.377321 22024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:44:53.382121 22024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:44:53.390219 22024 catalog_manager.cc:1347] Generated new cluster ID: d06db25f7f6e4705b63b1232bb589422
I20260619 06:44:53.390404 22024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:44:53.409376 22024 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:44:53.410596 22024 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:44:53.418313 22024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01fb65ef4264498b8707db1ff9987e2e: Generated new TSK 0
I20260619 06:44:53.419337 22024 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 06:44:53.464644 22040 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 06:44:53.464771 22040 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 06:44:53.466038 22013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:53.445657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58112 (local address 127.16.79.126:46029)
0619 06:44:53.445843 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:53.445850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:53.446363 (+ 513us) server_negotiation.cc:407] Connection header received
0619 06:44:53.446573 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:53.446580 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:53.446674 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:53.446840 (+ 166us) server_negotiation.cc:226] Negotiated authn=JWT
0619 06:44:53.447251 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:53.448430 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:53.449730 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:53.450443 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:53.450565 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:53.451018 (+ 453us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 06:44:53.465358 (+ 14340us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 06:44:53.465745 (+ 387us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58112: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":59}
W20260619 06:44:53.466056 22039 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:44:53.445631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46029 (local address 127.0.0.1:58112)
0619 06:44:53.446263 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:53.446282 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:53.446397 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:53.446858 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:53.446861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:53.446870 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0619 06:44:53.447102 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:53.447110 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:53.448575 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:53.448579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:53.449600 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:53.449611 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:53.450612 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:53.450616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:53.450743 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:53.450775 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 06:44:53.465595 (+ 14820us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 06:44:53.465785 (+ 190us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.79.126:46029: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":349,"threads_started":1}
W20260619 06:44:53.466156 22013 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58112: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44855/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260619 06:44:53.467765 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 21944
I20260619 06:44:53.468879 22005 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:53.561303 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 21944
2026-06-19T06:44:53Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (734 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 06:44:53.613178 16701 mini_oidc.cc:152] Starting JWKS server
I20260619 06:44:53.615571 16701 webserver.cc:460] Webserver started at https://127.0.0.1:34549/ using document root <none> and password file <none>
I20260619 06:44:53.615758 16701 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 06:44:53.616364 16701 webserver.cc:460] Webserver started at http://127.0.0.1:39061/ using document root <none> and password file <none>
2026-06-19T06:44:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:44:53Z Disabled control of system clock
I20260619 06:44:53.641618 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:45263
--webserver_interface=127.16.79.126
--webserver_port=0
--builtin_ntp_servers=127.16.79.84:33651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:45263
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34549/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 06:44:53.956291 22047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:53.956573 22047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:53.956657 22047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:53.963884 22047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:44:53.963996 22047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:53.964033 22047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:53.964102 22047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:44:53.964193 22047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 06:44:53.964273 22047 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 06:44:53.964313 22047 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34549/jwks.json
I20260619 06:44:53.973407 22047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:33651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:45263
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:45263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34549/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:53.974653 22047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:53.976372 22047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:53.987824 22055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:53.987794 22053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:53.988293 22052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:53.988903 22047 server_base.cc:1034] running on GCE node
I20260619 06:44:53.989627 22047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:53.990962 22047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:53.992228 22047 hybrid_clock.cc:648] HybridClock initialized: now 1781851493992198 us; error 57 us; skew 500 ppm
I20260619 06:44:53.992568 22047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:44:53.994666 22047 webserver.cc:460] Webserver started at http://127.16.79.126:42153/ using document root <none> and password file <none>
I20260619 06:44:53.995353 22047 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:53.995481 22047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:53.995793 22047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:53.998587 22047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "86b8715ad6e64ed8947917691bb6f3da"
format_stamp: "Formatted at 2026-06-19 06:44:53 on dist-test-slave-5w2x"
server_key: "db4371bd3a6e0b4592be50f52e3532ac"
server_key_iv: "6d839839bc376a962181ea52f0df4311"
server_key_version: "encryptionkey@0"
I20260619 06:44:53.999365 22047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "86b8715ad6e64ed8947917691bb6f3da"
format_stamp: "Formatted at 2026-06-19 06:44:53 on dist-test-slave-5w2x"
server_key: "db4371bd3a6e0b4592be50f52e3532ac"
server_key_iv: "6d839839bc376a962181ea52f0df4311"
server_key_version: "encryptionkey@0"
I20260619 06:44:54.004194 22047 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260619 06:44:54.007387 22061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:54.008716 22047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 06:44:54.008889 22047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "86b8715ad6e64ed8947917691bb6f3da"
format_stamp: "Formatted at 2026-06-19 06:44:53 on dist-test-slave-5w2x"
server_key: "db4371bd3a6e0b4592be50f52e3532ac"
server_key_iv: "6d839839bc376a962181ea52f0df4311"
server_key_version: "encryptionkey@0"
I20260619 06:44:54.009109 22047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:54.021296 22047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:54.022065 22047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:54.022365 22047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:54.040051 22047 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:45263
I20260619 06:44:54.040068 22112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:45263 every 8 connection(s)
I20260619 06:44:54.041411 22047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:44:54.045073 22113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:54.048669 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 22047
I20260619 06:44:54.048887 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:44:54.049504 16701 external_mini_cluster.cc:1428] Setting key f1695b971044216fb8947adf041f1886
I20260619 06:44:54.055876 22113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da: Bootstrap starting.
I20260619 06:44:54.059329 22113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:54.060640 22113 log.cc:824] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:54.063460 22116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:54.051374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34424 (local address 127.16.79.126:45263)
0619 06:44:54.052421 (+ 1047us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:54.052431 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:54.052470 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:44:54.054366 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:54.054387 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:54.054825 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:54.055123 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:54.055715 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:54.057118 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:54.058937 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:54.059688 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:54.059819 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:54.060963 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:54.060970 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:54.060984 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:54.061778 (+ 794us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:54.061889 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:54.061915 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:54.062029 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:54.062251 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 06:44:54.062572 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":367,"threads_started":1}
I20260619 06:44:54.063714 22113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da: No bootstrap required, opened a new log
I20260619 06:44:54.067101 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:44:54.071439 22113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b8715ad6e64ed8947917691bb6f3da" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45263 } }
I20260619 06:44:54.071805 22113 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:54.071913 22113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86b8715ad6e64ed8947917691bb6f3da, State: Initialized, Role: FOLLOWER
I20260619 06:44:54.072503 22113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b8715ad6e64ed8947917691bb6f3da" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45263 } }
I20260619 06:44:54.072742 22113 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:44:54.072877 22113 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:44:54.073055 22113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:54.075187 22113 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b8715ad6e64ed8947917691bb6f3da" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45263 } }
I20260619 06:44:54.075727 22113 leader_election.cc:302] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86b8715ad6e64ed8947917691bb6f3da; no voters:
I20260619 06:44:54.076503 22113 leader_election.cc:288] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:44:54.076634 22118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:54.077312 22118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [term 1 LEADER]: Becoming Leader. State: Replica: 86b8715ad6e64ed8947917691bb6f3da, State: Running, Role: LEADER
I20260619 06:44:54.077836 22118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b8715ad6e64ed8947917691bb6f3da" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45263 } }
I20260619 06:44:54.078670 22113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:44:54.080683 22119 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86b8715ad6e64ed8947917691bb6f3da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b8715ad6e64ed8947917691bb6f3da" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45263 } } }
I20260619 06:44:54.081139 22119 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:54.081498 22120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86b8715ad6e64ed8947917691bb6f3da. Latest consensus state: current_term: 1 leader_uuid: "86b8715ad6e64ed8947917691bb6f3da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b8715ad6e64ed8947917691bb6f3da" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45263 } } }
I20260619 06:44:54.082276 22120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:54.082631 22126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:44:54.087996 22126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:44:54.097414 22126 catalog_manager.cc:1347] Generated new cluster ID: 7fcaa43f30e94bdc96ee925fae3a321d
I20260619 06:44:54.097536 22126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:44:54.134228 22126 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:44:54.135586 22126 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:44:54.148173 22126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86b8715ad6e64ed8947917691bb6f3da: Generated new TSK 0
I20260619 06:44:54.149039 22126 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:44:54.178407 22116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:54.169835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34426 (local address 127.16.79.126:45263)
0619 06:44:54.170033 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:54.170040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:54.170500 (+ 460us) server_negotiation.cc:407] Connection header received
0619 06:44:54.170718 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:54.170724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:54.170826 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:54.170998 (+ 172us) server_negotiation.cc:226] Negotiated authn=JWT
0619 06:44:54.171440 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:54.172641 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:54.173970 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:54.174687 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:54.174810 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:54.175291 (+ 481us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 06:44:54.177737 (+ 2446us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34549/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 06:44:54.178063 (+ 326us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34426: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34549/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":64}
W20260619 06:44:54.178534 22116 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34426: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34549/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 06:44:54.180267 22142 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:44:54.169678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:45263 (local address 127.0.0.1:34426)
0619 06:44:54.170402 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:54.170418 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:54.170536 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:54.171008 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:54.171012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:54.171021 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0619 06:44:54.171302 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:54.171309 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:54.172784 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:54.172788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:54.173831 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:54.173842 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:54.174861 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:54.174866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:54.174996 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:54.175027 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 06:44:54.177955 (+ 2928us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34549/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 06:44:54.179801 (+ 1846us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.79.126:45263: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34549/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":1522,"thread_start_us":439,"threads_started":1}
I20260619 06:44:54.181339 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 22047
I20260619 06:44:54.182608 22108 generic_service.cc:196] Checking for leaks (request via RPC)
E20260619 06:44:54.190126 22143 webserver.cc:533] Webserver: sslize failed:
E20260619 06:44:54.190217 22143 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 06:44:54.276110 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 22047
2026-06-19T06:44:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (717 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T06:44:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:44:54Z Disabled control of system clock
I20260619 06:44:54.357172 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:42447
--webserver_interface=127.16.79.126
--webserver_port=0
--builtin_ntp_servers=127.16.79.84:34383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:42447
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/testchainca.cert with env {}
W20260619 06:44:54.679402 22148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:54.679713 22148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:54.679797 22148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:54.687242 22148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:44:54.687371 22148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:54.687409 22148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:54.687462 22148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:44:54.687507 22148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 06:44:54.687563 22148 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 06:44:54.687605 22148 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 06:44:54.698060 22148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:34383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:42447
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:42447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:54.699420 22148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:54.700735 22148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:54.713065 22153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:54.713111 22154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:54.713141 22156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:54.714233 22148 server_base.cc:1034] running on GCE node
I20260619 06:44:54.715150 22148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:54.716796 22148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:54.718061 22148 hybrid_clock.cc:648] HybridClock initialized: now 1781851494717999 us; error 64 us; skew 500 ppm
I20260619 06:44:54.718477 22148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:44:54.720824 22148 webserver.cc:460] Webserver started at http://127.16.79.126:35957/ using document root <none> and password file <none>
I20260619 06:44:54.721536 22148 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:54.721675 22148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:54.721997 22148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:54.725035 22148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "24bb7c2ebdb24975a81060e346006f0d"
format_stamp: "Formatted at 2026-06-19 06:44:54 on dist-test-slave-5w2x"
server_key: "95efcd232b60fce90f24b19e35544c33"
server_key_iv: "9ff47e0a02849be5482f659dc1aac68d"
server_key_version: "encryptionkey@0"
I20260619 06:44:54.725878 22148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "24bb7c2ebdb24975a81060e346006f0d"
format_stamp: "Formatted at 2026-06-19 06:44:54 on dist-test-slave-5w2x"
server_key: "95efcd232b60fce90f24b19e35544c33"
server_key_iv: "9ff47e0a02849be5482f659dc1aac68d"
server_key_version: "encryptionkey@0"
I20260619 06:44:54.730893 22148 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:44:54.734516 22162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:54.736119 22148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260619 06:44:54.736415 22148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "24bb7c2ebdb24975a81060e346006f0d"
format_stamp: "Formatted at 2026-06-19 06:44:54 on dist-test-slave-5w2x"
server_key: "95efcd232b60fce90f24b19e35544c33"
server_key_iv: "9ff47e0a02849be5482f659dc1aac68d"
server_key_version: "encryptionkey@0"
I20260619 06:44:54.736656 22148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:54.748665 22148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:54.749478 22148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:54.749776 22148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:54.768203 22148 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:42447
I20260619 06:44:54.768260 22213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:42447 every 8 connection(s)
I20260619 06:44:54.769524 22148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:44:54.773212 22214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:44:54.776110 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 22148
I20260619 06:44:54.776350 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:44:54.776998 16701 external_mini_cluster.cc:1428] Setting key bfc5e709014ad6c3250e9bb41f7e6619
I20260619 06:44:54.786366 22214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d: Bootstrap starting.
I20260619 06:44:54.790422 22214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:54.791707 22214 log.cc:824] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:54.791924 22217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:54.778862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54116 (local address 127.16.79.126:42447)
0619 06:44:54.779916 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:54.779930 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:54.779982 (+ 52us) server_negotiation.cc:407] Connection header received
0619 06:44:54.781947 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:54.781982 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:54.782614 (+ 632us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:54.783080 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:54.783613 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:54.785573 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:54.786831 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:54.787816 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:54.788027 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:54.789124 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:54.789134 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:54.789150 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:44:54.789856 (+ 706us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:54.789968 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:44:54.789997 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:54.790124 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:54.790302 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 06:44:54.790644 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":315,"threads_started":1}
I20260619 06:44:54.795710 22214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d: No bootstrap required, opened a new log
I20260619 06:44:54.795827 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:44:54.803709 22214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24bb7c2ebdb24975a81060e346006f0d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42447 } }
I20260619 06:44:54.804035 22214 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:54.804102 22214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24bb7c2ebdb24975a81060e346006f0d, State: Initialized, Role: FOLLOWER
I20260619 06:44:54.804700 22214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24bb7c2ebdb24975a81060e346006f0d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42447 } }
I20260619 06:44:54.804908 22214 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:44:54.805007 22214 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:44:54.805121 22214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:54.807142 22214 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24bb7c2ebdb24975a81060e346006f0d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42447 } }
I20260619 06:44:54.807582 22214 leader_election.cc:302] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24bb7c2ebdb24975a81060e346006f0d; no voters:
I20260619 06:44:54.808183 22214 leader_election.cc:288] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:44:54.808344 22219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:54.809432 22219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [term 1 LEADER]: Becoming Leader. State: Replica: 24bb7c2ebdb24975a81060e346006f0d, State: Running, Role: LEADER
I20260619 06:44:54.809898 22219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24bb7c2ebdb24975a81060e346006f0d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42447 } }
I20260619 06:44:54.810321 22214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:44:54.812716 22220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24bb7c2ebdb24975a81060e346006f0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24bb7c2ebdb24975a81060e346006f0d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42447 } } }
I20260619 06:44:54.812793 22221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24bb7c2ebdb24975a81060e346006f0d. Latest consensus state: current_term: 1 leader_uuid: "24bb7c2ebdb24975a81060e346006f0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24bb7c2ebdb24975a81060e346006f0d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42447 } } }
I20260619 06:44:54.813109 22220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:54.813144 22221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:54.815003 22225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:44:54.820350 22225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:44:54.830485 22225 catalog_manager.cc:1347] Generated new cluster ID: fb5bf69b53f64404b815b4a6029fe486
I20260619 06:44:54.831115 22225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:44:54.839422 22225 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:44:54.840899 22225 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:44:54.847802 22225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24bb7c2ebdb24975a81060e346006f0d: Generated new TSK 0
I20260619 06:44:54.848604 22225 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:44:54.939147 22217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:54.863827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54124 (local address 127.16.79.126:42447)
0619 06:44:54.864050 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:54.864057 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:54.864611 (+ 554us) server_negotiation.cc:407] Connection header received
0619 06:44:54.864857 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:54.864860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:54.864939 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:54.865052 (+ 113us) server_negotiation.cc:226] Negotiated authn=JWT
0619 06:44:54.865564 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:54.866247 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:54.868278 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:54.868707 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:54.868797 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:54.869324 (+ 527us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 06:44:54.938610 (+ 69286us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 06:44:54.938925 (+ 315us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54124: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":96}
W20260619 06:44:54.939319 22217 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54124: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 06:44:54.940009 22243 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:44:54.863628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:42447 (local address 127.0.0.1:54124)
0619 06:44:54.864496 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:54.864517 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:54.864678 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:54.865101 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:54.865106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:54.865119 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0619 06:44:54.865412 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:54.865424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:54.866392 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:54.866398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:54.868120 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:54.868161 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:54.868871 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:54.868878 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:54.869055 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:54.869107 (+ 52us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 06:44:54.938895 (+ 69788us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 06:44:54.939213 (+ 318us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.16.79.126:42447: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":573,"threads_started":1}
I20260619 06:44:54.941480 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 22148
I20260619 06:44:54.942584 22209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:55.039814 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 22148
2026-06-19T06:44:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (743 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (403 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (402 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:44:55 dist-test-slave-5w2x krb5kdc[22258](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:44:55 dist-test-slave-5w2x krb5kdc[22258](info): set up 2 sockets
Jun 19 06:44:55 dist-test-slave-5w2x krb5kdc[22258](info): commencing operation
krb5kdc: starting...
W20260619 06:44:56.928750 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.006s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:44:57 dist-test-slave-5w2x krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:44:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:44:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:44:57.124086 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:44667
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:44667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:57.447060 22271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:57.447352 22271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:57.447443 22271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:57.454671 22271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:44:57.454782 22271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:57.454903 22271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:57.455005 22271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:44:57.455103 22271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:44:57.464933 22271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:44667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:44667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:57.466289 22271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:57.467607 22271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:57.480155 22277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:57.480150 22276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:57.480187 22279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:57.481045 22271 server_base.cc:1034] running on GCE node
I20260619 06:44:57.481966 22271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:57.483342 22271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:57.484560 22271 hybrid_clock.cc:648] HybridClock initialized: now 1781851497484492 us; error 60 us; skew 500 ppm
I20260619 06:44:57.484879 22271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:57 dist-test-slave-5w2x krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851497, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:57.487864 22271 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:44:57.489326 22271 webserver.cc:460] Webserver started at http://127.16.79.126:45011/ using document root <none> and password file <none>
I20260619 06:44:57.490006 22271 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:57.490147 22271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:57.490469 22271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:57.493461 22271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "bcfd2c551ac440278325822dbcf77e91"
format_stamp: "Formatted at 2026-06-19 06:44:57 on dist-test-slave-5w2x"
server_key: "3b522207aa7028958811a87c6905768c"
server_key_iv: "5e6d5abc14703d3aa9d66a701e585f8d"
server_key_version: "encryptionkey@0"
I20260619 06:44:57.494277 22271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "bcfd2c551ac440278325822dbcf77e91"
format_stamp: "Formatted at 2026-06-19 06:44:57 on dist-test-slave-5w2x"
server_key: "3b522207aa7028958811a87c6905768c"
server_key_iv: "5e6d5abc14703d3aa9d66a701e585f8d"
server_key_version: "encryptionkey@0"
I20260619 06:44:57.499295 22271 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:44:57.502710 22286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:57.504029 22271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 06:44:57.504247 22271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "bcfd2c551ac440278325822dbcf77e91"
format_stamp: "Formatted at 2026-06-19 06:44:57 on dist-test-slave-5w2x"
server_key: "3b522207aa7028958811a87c6905768c"
server_key_iv: "5e6d5abc14703d3aa9d66a701e585f8d"
server_key_version: "encryptionkey@0"
I20260619 06:44:57.504417 22271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:57.518043 22271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:57.522265 22271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:57.522543 22271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:57.540423 22271 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:44667
I20260619 06:44:57.540453 22337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:44667 every 8 connection(s)
I20260619 06:44:57.541786 22271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:44:57.543151 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 22271
I20260619 06:44:57.543356 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:44:57.544193 16701 external_mini_cluster.cc:1428] Setting key 1178082d805a02bfa23b8256432f5ca6
I20260619 06:44:57.546553 22338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 06:44:57 dist-test-slave-5w2x krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:57.559135 22338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91: Bootstrap starting.
I20260619 06:44:57.563128 22338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91: Neither blocks nor log segments found. Creating new log.
I20260619 06:44:57.564407 22338 log.cc:824] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91: Log is configured to *not* fsync() on all Append() calls
I20260619 06:44:57.567294 22338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91: No bootstrap required, opened a new log
I20260619 06:44:57.567799 22341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:57.546198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56276 (local address 127.16.79.126:44667)
0619 06:44:57.546994 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:57.547009 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:57.547055 (+ 46us) server_negotiation.cc:407] Connection header received
0619 06:44:57.549317 (+ 2262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:57.549342 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:57.549847 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:57.550277 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:57.551567 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:57.553629 (+ 2062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:57.554835 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:57.555860 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:57.556034 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:57.559900 (+ 3866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:57.559909 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:57.559926 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:57.559972 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:57.563796 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:57.564507 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:57.564516 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:57.564524 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:57.564736 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:57.565122 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:57.565129 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:57.565132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:57.565669 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:57.565964 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:57.566386 (+ 422us) server_negotiation.cc:299] Negotiation successful
0619 06:44:57.566785 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":373,"threads_started":1}
I20260619 06:44:57.574860 22338 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfd2c551ac440278325822dbcf77e91" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44667 } }
I20260619 06:44:57.575229 22338 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:44:57.575345 22338 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcfd2c551ac440278325822dbcf77e91, State: Initialized, Role: FOLLOWER
I20260619 06:44:57.575971 22338 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfd2c551ac440278325822dbcf77e91" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44667 } }
I20260619 06:44:57.576246 22338 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:44:57.576424 22338 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:44:57.576608 22338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:44:57.578694 22338 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfd2c551ac440278325822dbcf77e91" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44667 } }
I20260619 06:44:57.579176 22338 leader_election.cc:302] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcfd2c551ac440278325822dbcf77e91; no voters:
I20260619 06:44:57.579706 22338 leader_election.cc:288] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:44:57.579840 22343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:44:57.580618 22343 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [term 1 LEADER]: Becoming Leader. State: Replica: bcfd2c551ac440278325822dbcf77e91, State: Running, Role: LEADER
I20260619 06:44:57.581137 22343 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfd2c551ac440278325822dbcf77e91" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44667 } }
I20260619 06:44:57.581826 22338 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:44:57.583811 22344 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcfd2c551ac440278325822dbcf77e91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfd2c551ac440278325822dbcf77e91" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44667 } } }
I20260619 06:44:57.584231 22344 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:57.584370 22345 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcfd2c551ac440278325822dbcf77e91. Latest consensus state: current_term: 1 leader_uuid: "bcfd2c551ac440278325822dbcf77e91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcfd2c551ac440278325822dbcf77e91" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 44667 } } }
I20260619 06:44:57.585253 22345 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91 [sys.catalog]: This master's current role is: LEADER
I20260619 06:44:57.587796 22349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:44:57.593007 22349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:44:57.601476 22349 catalog_manager.cc:1347] Generated new cluster ID: ed83a75df9684721ac3f3eac5877c883
I20260619 06:44:57.601625 22349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:44:57.614408 22349 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:44:57.615665 22349 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:44:57.626641 22349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcfd2c551ac440278325822dbcf77e91: Generated new TSK 0
I20260619 06:44:57.627485 22349 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:44:57.673713 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:44667
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:57.999609 22364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:57.999879 22364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:58.000058 22364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:58.007382 22364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:58.007516 22364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:58.007700 22364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:44:58.017493 22364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:44667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:58.018853 22364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:58.020216 22364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:58.033471 22369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:58.033535 22370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:58.033495 22372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:58.034274 22364 server_base.cc:1034] running on GCE node
I20260619 06:44:58.034897 22364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:58.035794 22364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:58.037010 22364 hybrid_clock.cc:648] HybridClock initialized: now 1781851498036974 us; error 54 us; skew 500 ppm
I20260619 06:44:58.037334 22364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:58 dist-test-slave-5w2x krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851498, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:58.040437 22364 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:44:58.041836 22364 webserver.cc:460] Webserver started at http://127.16.79.65:41315/ using document root <none> and password file <none>
I20260619 06:44:58.042515 22364 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:58.042659 22364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:58.042960 22364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:58.045838 22364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "7db6cbf3f85845a3b26ef9bc95c13794"
format_stamp: "Formatted at 2026-06-19 06:44:58 on dist-test-slave-5w2x"
server_key: "8e27b83f0b7ac4e43baabb706b238c45"
server_key_iv: "983ca5efb9e065193bbc4fa120903724"
server_key_version: "encryptionkey@0"
I20260619 06:44:58.046584 22364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "7db6cbf3f85845a3b26ef9bc95c13794"
format_stamp: "Formatted at 2026-06-19 06:44:58 on dist-test-slave-5w2x"
server_key: "8e27b83f0b7ac4e43baabb706b238c45"
server_key_iv: "983ca5efb9e065193bbc4fa120903724"
server_key_version: "encryptionkey@0"
I20260619 06:44:58.051448 22364 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 06:44:58.054898 22379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:58.056268 22364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 06:44:58.056438 22364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "7db6cbf3f85845a3b26ef9bc95c13794"
format_stamp: "Formatted at 2026-06-19 06:44:58 on dist-test-slave-5w2x"
server_key: "8e27b83f0b7ac4e43baabb706b238c45"
server_key_iv: "983ca5efb9e065193bbc4fa120903724"
server_key_version: "encryptionkey@0"
I20260619 06:44:58.056641 22364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:58.077263 22364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:58.081207 22364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:58.081498 22364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:58.082489 22364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:58.083779 22364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:58.083880 22364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:58.084017 22364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:58.084087 22364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:58.116581 22364 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:40143
I20260619 06:44:58.116624 22491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:40143 every 8 connection(s)
I20260619 06:44:58.117831 22364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:44:58.125193 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 22364
I20260619 06:44:58.125387 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:44:58.126156 16701 external_mini_cluster.cc:1428] Setting key a40d92152150eece1180915a4109a66f
Jun 19 06:44:58 dist-test-slave-5w2x krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851498, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:58.138351 22494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:58.120209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:51963 (local address 127.16.79.126:44667)
0619 06:44:58.120627 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:58.120633 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:58.121657 (+ 1024us) server_negotiation.cc:407] Connection header received
0619 06:44:58.123338 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:58.123342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:58.123404 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:58.123497 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:58.124997 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:58.126215 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:58.128253 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:58.128645 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:58.128791 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:58.132818 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:58.132823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:58.132827 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:58.132858 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:58.135368 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:58.136091 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:58.136100 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:58.136103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:58.136263 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:58.136676 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:58.136681 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:58.136684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:58.136944 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:58.137094 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:58.137781 (+ 687us) server_negotiation.cc:299] Negotiation successful
0619 06:44:58.137986 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":216,"threads_started":1}
I20260619 06:44:58.139321 22495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:58.120811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:44667 (local address 127.16.79.65:51963)
0619 06:44:58.121494 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:58.121532 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:58.123096 (+ 1564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:58.123697 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:58.123705 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:58.124280 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:58.124818 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:58.124839 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:58.126435 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:58.126441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:58.128061 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:58.128073 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:58.128862 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:58.128870 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:58.129112 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:58.129836 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:58.129870 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:58.132583 (+ 2713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:58.135510 (+ 2927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:58.135519 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:58.135536 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:58.135954 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:58.136367 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:58.136372 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:58.136376 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:58.136554 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:58.137069 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:58.137076 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:58.137420 (+ 344us) client_negotiation.cc:769] Sending connection context
0619 06:44:58.137687 (+ 267us) client_negotiation.cc:240] Negotiation successful
0619 06:44:58.138242 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":268,"threads_started":1}
I20260619 06:44:58.140885 22492 heartbeater.cc:347] Connected to a master server at 127.16.79.126:44667
I20260619 06:44:58.141234 22492 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:58.142118 22492 heartbeater.cc:511] Master 127.16.79.126:44667 requested a full tablet report, sending...
I20260619 06:44:58.144563 22303 ts_manager.cc:194] Registered new tserver with Master: 7db6cbf3f85845a3b26ef9bc95c13794 (127.16.79.65:40143)
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
I20260619 06:44:58.146642 22303 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:51963
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:44:58.170903 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:44667
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:58.489450 22498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:58.489738 22498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:58.489883 22498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:58.497134 22498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:58.497256 22498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:58.497470 22498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:44:58.507810 22498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:44667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:58.509356 22498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:58.510711 22498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:58.523924 22503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:58.523869 22504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:58.523945 22506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:58.524806 22498 server_base.cc:1034] running on GCE node
I20260619 06:44:58.525415 22498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:58.526389 22498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:58.527632 22498 hybrid_clock.cc:648] HybridClock initialized: now 1781851498527569 us; error 56 us; skew 500 ppm
I20260619 06:44:58.527952 22498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:58 dist-test-slave-5w2x krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851498, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:58.534693 22498 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:44:58.536335 22498 webserver.cc:460] Webserver started at http://127.16.79.66:40939/ using document root <none> and password file <none>
I20260619 06:44:58.537081 22498 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:58.537197 22498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:58.537510 22498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:58.540644 22498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "08c919fc53594bdb9524f1aed7ef3891"
format_stamp: "Formatted at 2026-06-19 06:44:58 on dist-test-slave-5w2x"
server_key: "a74d2b21af7ebec8fc2c9318d2ca263d"
server_key_iv: "edd8b8140600f04c795410b530bb9214"
server_key_version: "encryptionkey@0"
I20260619 06:44:58.541445 22498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "08c919fc53594bdb9524f1aed7ef3891"
format_stamp: "Formatted at 2026-06-19 06:44:58 on dist-test-slave-5w2x"
server_key: "a74d2b21af7ebec8fc2c9318d2ca263d"
server_key_iv: "edd8b8140600f04c795410b530bb9214"
server_key_version: "encryptionkey@0"
I20260619 06:44:58.546890 22498 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:44:58.550732 22513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:58.552273 22498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 06:44:58.552450 22498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "08c919fc53594bdb9524f1aed7ef3891"
format_stamp: "Formatted at 2026-06-19 06:44:58 on dist-test-slave-5w2x"
server_key: "a74d2b21af7ebec8fc2c9318d2ca263d"
server_key_iv: "edd8b8140600f04c795410b530bb9214"
server_key_version: "encryptionkey@0"
I20260619 06:44:58.552655 22498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:58.574369 22498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:58.578909 22498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:58.579226 22498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:58.580313 22498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:58.581688 22498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:58.581791 22498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:58.581934 22498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:58.582012 22498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:58.614768 22498 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:46345
I20260619 06:44:58.614822 22625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:46345 every 8 connection(s)
I20260619 06:44:58.616097 22498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:44:58.619650 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 22498
I20260619 06:44:58.619832 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:44:58.620416 16701 external_mini_cluster.cc:1428] Setting key 8d67010b855494e2d606b932f8e00c17
Jun 19 06:44:58 dist-test-slave-5w2x krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851498, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:58.637298 22494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:58.618691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:34187 (local address 127.16.79.126:44667)
0619 06:44:58.618866 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:58.618870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:58.620125 (+ 1255us) server_negotiation.cc:407] Connection header received
0619 06:44:58.622117 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:58.622124 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:58.622198 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:58.622275 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:58.623769 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:58.624967 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:58.626593 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:58.627330 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:58.627476 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:58.631581 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:58.631586 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:58.631590 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:58.631621 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:58.634179 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:58.634953 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:58.634964 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:58.634966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:58.635091 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:58.635509 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:58.635516 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:58.635519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:58.635786 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:58.635934 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:58.636698 (+ 764us) server_negotiation.cc:299] Negotiation successful
0619 06:44:58.636948 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 06:44:58.638329 22628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:58.619093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:44667 (local address 127.16.79.66:34187)
0619 06:44:58.619945 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:58.619990 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:58.621854 (+ 1864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:58.622540 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:58.622548 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:58.623042 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:58.623590 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:58.623606 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:58.625137 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:58.625141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:58.626441 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:58.626453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:58.627596 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:58.627601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:58.627811 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:58.628609 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:58.628650 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:58.631329 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:58.634351 (+ 3022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:58.634361 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:58.634380 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:58.634818 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:58.635206 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:58.635211 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:58.635215 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:58.635402 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:58.635924 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:58.635932 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:58.636346 (+ 414us) client_negotiation.cc:769] Sending connection context
0619 06:44:58.636620 (+ 274us) client_negotiation.cc:240] Negotiation successful
0619 06:44:58.637209 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":387,"threads_started":1}
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
I20260619 06:44:58.639991 22626 heartbeater.cc:347] Connected to a master server at 127.16.79.126:44667
I20260619 06:44:58.640377 22626 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:58.641083 22626 heartbeater.cc:511] Master 127.16.79.126:44667 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
I20260619 06:44:58.642671 22303 ts_manager.cc:194] Registered new tserver with Master: 08c919fc53594bdb9524f1aed7ef3891 (127.16.79.66:46345)
I20260619 06:44:58.643460 22303 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:34187
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:44:58.665266 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:44667
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:44:58.981959 22631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:44:58.982242 22631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:44:58.982414 22631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:44:58.989737 22631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:44:58.989845 22631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:44:58.989954 22631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:44:58.999421 22631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:44667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:44:59.000645 22631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:44:59.001876 22631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:44:59.014549 22636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:59.014498 22637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:44:59.014518 22639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:44:59.015508 22631 server_base.cc:1034] running on GCE node
I20260619 06:44:59.016088 22631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:44:59.016988 22631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:44:59.018188 22631 hybrid_clock.cc:648] HybridClock initialized: now 1781851499018140 us; error 41 us; skew 500 ppm
I20260619 06:44:59.018507 22631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851499, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:59.021538 22631 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:44:59.022928 22631 webserver.cc:460] Webserver started at http://127.16.79.67:45985/ using document root <none> and password file <none>
I20260619 06:44:59.023595 22631 fs_manager.cc:359] Metadata directory not provided
I20260619 06:44:59.023731 22631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:44:59.024019 22631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:44:59.026871 22631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "9dde27ab7adb4bfbac2e2dd26e341fba"
format_stamp: "Formatted at 2026-06-19 06:44:59 on dist-test-slave-5w2x"
server_key: "595865b8e5c9bf22b7732410a95ae74d"
server_key_iv: "8f6c706f68f25e80c0bf84fb88e2ae55"
server_key_version: "encryptionkey@0"
I20260619 06:44:59.027647 22631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "9dde27ab7adb4bfbac2e2dd26e341fba"
format_stamp: "Formatted at 2026-06-19 06:44:59 on dist-test-slave-5w2x"
server_key: "595865b8e5c9bf22b7732410a95ae74d"
server_key_iv: "8f6c706f68f25e80c0bf84fb88e2ae55"
server_key_version: "encryptionkey@0"
I20260619 06:44:59.032985 22631 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260619 06:44:59.036536 22646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:59.037883 22631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:44:59.038075 22631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "9dde27ab7adb4bfbac2e2dd26e341fba"
format_stamp: "Formatted at 2026-06-19 06:44:59 on dist-test-slave-5w2x"
server_key: "595865b8e5c9bf22b7732410a95ae74d"
server_key_iv: "8f6c706f68f25e80c0bf84fb88e2ae55"
server_key_version: "encryptionkey@0"
I20260619 06:44:59.038321 22631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:44:59.058996 22631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:44:59.063102 22631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:44:59.063380 22631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:44:59.064383 22631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:44:59.065663 22631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:44:59.065764 22631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:59.065907 22631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:44:59.065977 22631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:44:59.100411 22631 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:36797
I20260619 06:44:59.100500 22758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:36797 every 8 connection(s)
I20260619 06:44:59.101728 22631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:44:59.104727 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 22631
I20260619 06:44:59.104895 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:44:59.105367 16701 external_mini_cluster.cc:1428] Setting key 73724f92cfe395089d590e3a8370cd67
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851499, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:44:59.122874 22494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:59.104280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:58663 (local address 127.16.79.126:44667)
0619 06:44:59.104497 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:59.104504 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:59.106005 (+ 1501us) server_negotiation.cc:407] Connection header received
0619 06:44:59.108096 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:59.108101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:59.108220 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:59.108301 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:59.109736 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:59.110471 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:59.112066 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:59.112535 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:59.112627 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:59.116032 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:59.116039 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:59.116043 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:59.116083 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:59.119480 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:59.120466 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:59.120472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:59.120476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:59.120638 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:59.121117 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:59.121120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:59.121123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:59.121408 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:59.121520 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:59.122472 (+ 952us) server_negotiation.cc:299] Negotiation successful
0619 06:44:59.122636 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260619 06:44:59.123873 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:59.104965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:44667 (local address 127.16.79.67:58663)
0619 06:44:59.105818 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0619 06:44:59.105868 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 06:44:59.107778 (+ 1910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:44:59.108531 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:44:59.108540 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:44:59.109044 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:44:59.109590 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:59.109608 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:59.110634 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:59.110638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:59.111904 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:44:59.111917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:44:59.112732 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:44:59.112737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:44:59.112970 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:59.113587 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:44:59.113610 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:44:59.115779 (+ 2169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:44:59.119654 (+ 3875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:59.119663 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:59.119680 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:59.120327 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:59.120770 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:44:59.120775 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:44:59.120779 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:44:59.120996 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:44:59.121564 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:44:59.121571 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:44:59.122002 (+ 431us) client_negotiation.cc:769] Sending connection context
0619 06:44:59.122308 (+ 306us) client_negotiation.cc:240] Negotiation successful
0619 06:44:59.122824 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":369,"threads_started":1}
I20260619 06:44:59.125202 22759 heartbeater.cc:347] Connected to a master server at 127.16.79.126:44667
I20260619 06:44:59.125543 22759 heartbeater.cc:464] Registering TS with master...
I20260619 06:44:59.126264 22759 heartbeater.cc:511] Master 127.16.79.126:44667 requested a full tablet report, sending...
I20260619 06:44:59.127724 22303 ts_manager.cc:194] Registered new tserver with Master: 9dde27ab7adb4bfbac2e2dd26e341fba (127.16.79.67:36797)
I20260619 06:44:59.128571 22303 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:58663
I20260619 06:44:59.133028 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:44:59.149782 22492 heartbeater.cc:503] Master 127.16.79.126:44667 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/test-admin.keytab.
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:59.159075 16701 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 06:44:59.164340 22766 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:44:59.161914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37072 (local address 127.16.79.65:40143)
0619 06:44:59.162535 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:59.162542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:59.162569 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:44:59.162639 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:59.162645 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:59.162861 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:59.163002 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:59.163872 (+ 870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37072: BlockingRecv error: recv got EOF from 127.0.0.1:37072 (error 108)
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":321,"threads_started":1}
W20260619 06:44:59.164377 16701 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/krb5cc' of type FILE
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:59.165980 16701 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:59.167351 16701 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:44:59.183287 22766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:59.168906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37078 (local address 127.16.79.65:40143)
0619 06:44:59.169109 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:59.169120 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:59.169368 (+ 248us) server_negotiation.cc:407] Connection header received
0619 06:44:59.169590 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:59.169594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:59.169657 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:59.169759 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:59.170812 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:59.171768 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:59.173218 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:59.173863 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:59.174024 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:59.177534 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:59.177545 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:59.177560 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:59.177596 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:59.180502 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:59.181138 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:59.181150 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:59.181153 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:59.181277 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:59.181647 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:59.181652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:59.181659 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:59.182074 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:59.182369 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:59.182734 (+ 365us) server_negotiation.cc:299] Negotiation successful
0619 06:44:59.182991 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260619 06:44:59.189671 22766 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:44:59.187789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37088 (local address 127.16.79.65:40143)
0619 06:44:59.187998 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:59.188003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:59.188318 (+ 315us) server_negotiation.cc:407] Connection header received
0619 06:44:59.188510 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:59.188515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:59.188585 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:59.188691 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:59.189411 (+ 720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37088: BlockingRecv error: recv got EOF from 127.0.0.1:37088 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260619 06:44:59.190068 16701 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/krb5cc)
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:59.191592 16701 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:59.192914 16701 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:44:59.205922 22766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:59.194143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37092 (local address 127.16.79.65:40143)
0619 06:44:59.194361 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:59.194367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:59.194575 (+ 208us) server_negotiation.cc:407] Connection header received
0619 06:44:59.194705 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:59.194709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:59.194766 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:59.194842 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:59.195694 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:59.196432 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:59.197539 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:59.197949 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:59.198049 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:59.201390 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:59.201394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:59.201397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:59.201427 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:59.203730 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:59.204363 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:59.204367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:59.204370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:59.204479 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:59.204783 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:59.204786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:59.204788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:59.205039 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:59.205218 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:59.205455 (+ 237us) server_negotiation.cc:299] Negotiation successful
0619 06:44:59.205640 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260619 06:44:59.210167 22766 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:44:59.208493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37094 (local address 127.16.79.65:40143)
0619 06:44:59.208702 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:59.208708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:59.209015 (+ 307us) server_negotiation.cc:407] Connection header received
0619 06:44:59.209128 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:59.209131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:59.209192 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:59.209266 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:59.209913 (+ 647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37094: BlockingRecv error: recv got EOF from 127.0.0.1:37094 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260619 06:44:59.210253 16701 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781851451399607-16701-0/krb5kdc/krb5cc' of type FILE
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:59.211783 16701 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22258](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:44:59.213050 16701 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:44:59.226776 22766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:59.214275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37108 (local address 127.16.79.65:40143)
0619 06:44:59.214450 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:59.214455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:59.214632 (+ 177us) server_negotiation.cc:407] Connection header received
0619 06:44:59.214819 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:59.214824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:59.214886 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:59.214979 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:59.215871 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:59.216643 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:59.217865 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:59.218269 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:59.218402 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:59.221787 (+ 3385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:59.221792 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:59.221795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:59.221824 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:59.224313 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:59.224907 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:59.224912 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:59.224916 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:59.225055 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:59.225416 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:59.225422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:59.225426 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:59.225704 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:59.226007 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:59.226266 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 06:44:59.226465 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:44:59.228375 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 22364
I20260619 06:44:59.240828 22766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:59.229108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37118 (local address 127.16.79.65:40143)
0619 06:44:59.229297 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:59.229304 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:59.229628 (+ 324us) server_negotiation.cc:407] Connection header received
0619 06:44:59.229810 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:59.229815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:59.229896 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:59.229994 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:59.230880 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:59.231552 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:59.233295 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:59.233702 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:59.233833 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:59.235805 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:59.235813 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:59.235822 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:59.235853 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:59.238069 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:59.238706 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:59.238711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:59.238714 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:59.238875 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:59.239297 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:59.239303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:59.239307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:59.239606 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:59.239850 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:59.240177 (+ 327us) server_negotiation.cc:299] Negotiation successful
0619 06:44:59.240387 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:44:59.241622 22487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:59.328392 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 22364
I20260619 06:44:59.344482 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 22498
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:44:59.360828 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:59.345432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49526 (local address 127.16.79.66:46345)
0619 06:44:59.345931 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:59.345940 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:59.345974 (+ 34us) server_negotiation.cc:407] Connection header received
0619 06:44:59.346039 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:59.346046 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:59.346274 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:59.346488 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:59.347525 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:59.348711 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:59.350093 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:59.350676 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:59.350825 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:59.355065 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:59.355077 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:59.355092 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:59.355128 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:59.358188 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:59.358852 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:59.358857 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:59.358860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:59.358972 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:59.359284 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:59.359286 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:59.359289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:59.359603 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:59.359836 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:59.360097 (+ 261us) server_negotiation.cc:299] Negotiation successful
0619 06:44:59.360531 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":252,"threads_started":1}
I20260619 06:44:59.362010 22621 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:59.445422 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 22498
I20260619 06:44:59.462368 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 22631
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22258](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:44:59.477818 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:44:59.463318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54482 (local address 127.16.79.67:36797)
0619 06:44:59.463959 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0619 06:44:59.463966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:44:59.463995 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:44:59.464054 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:44:59.464060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:44:59.464405 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:44:59.464569 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:44:59.465612 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:59.466612 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:59.467841 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:44:59.468460 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:44:59.468598 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:44:59.472422 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:44:59.472432 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:44:59.472446 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:44:59.472481 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:44:59.475261 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:59.475899 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:59.475906 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:59.475908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:59.476032 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:44:59.476432 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:44:59.476435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:44:59.476437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:44:59.476761 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:44:59.476983 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0619 06:44:59.477220 (+ 237us) server_negotiation.cc:299] Negotiation successful
0619 06:44:59.477508 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":342,"threads_started":1}
I20260619 06:44:59.479027 22754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:59.561121 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 22631
I20260619 06:44:59.576915 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 22271
I20260619 06:44:59.578018 22333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:44:59.678519 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 22271
2026-06-19T06:44:59Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3837 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22786](info): set up 2 sockets
Jun 19 06:44:59 dist-test-slave-5w2x krb5kdc[22786](info): commencing operation
krb5kdc: starting...
W20260619 06:45:00.776978 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.001s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:45:00 dist-test-slave-5w2x krb5kdc[22786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:45:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:45:00Z Disabled control of system clock
WARNING: no policy specified for oryx/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.126.keytab.
Entry for principal oryx/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.126.keytab.
I20260619 06:45:00.981217 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:41937
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.126.keytab
--principal=oryx/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:41937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:01.304440 22799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:01.304800 22799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:01.304982 22799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:01.312251 22799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:45:01.312398 22799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:01.312505 22799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:01.312609 22799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:45:01.312681 22799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:45:01.322917 22799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:41937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.126.keytab
--principal=oryx/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:41937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:01.324302 22799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:01.326308 22799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:01.338078 22804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:01.338023 22807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:01.338122 22805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:01.341543 22799 server_base.cc:1034] running on GCE node
I20260619 06:45:01.342311 22799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:01.343667 22799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:01.344867 22799 hybrid_clock.cc:648] HybridClock initialized: now 1781851501344826 us; error 29 us; skew 500 ppm
I20260619 06:45:01.345211 22799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:01 dist-test-slave-5w2x krb5kdc[22786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851501, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:01.348208 22799 init.cc:377] Logged in from keytab as oryx/127.16.79.126@KRBTEST.COM (short username oryx)
I20260619 06:45:01.349735 22799 webserver.cc:460] Webserver started at http://127.16.79.126:43155/ using document root <none> and password file <none>
I20260619 06:45:01.350392 22799 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:01.350487 22799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:01.350729 22799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:01.353659 22799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "eeb9870492ab483c95bd19f6f547aa11"
format_stamp: "Formatted at 2026-06-19 06:45:01 on dist-test-slave-5w2x"
server_key: "4d7a285d9b707f52c1d75525fa9ae561"
server_key_iv: "fa88c941924c8bdcd788361e2cd8cab5"
server_key_version: "encryptionkey@0"
I20260619 06:45:01.354385 22799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "eeb9870492ab483c95bd19f6f547aa11"
format_stamp: "Formatted at 2026-06-19 06:45:01 on dist-test-slave-5w2x"
server_key: "4d7a285d9b707f52c1d75525fa9ae561"
server_key_iv: "fa88c941924c8bdcd788361e2cd8cab5"
server_key_version: "encryptionkey@0"
I20260619 06:45:01.359440 22799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 06:45:01.362854 22814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:01.364213 22799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 06:45:01.364344 22799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "eeb9870492ab483c95bd19f6f547aa11"
format_stamp: "Formatted at 2026-06-19 06:45:01 on dist-test-slave-5w2x"
server_key: "4d7a285d9b707f52c1d75525fa9ae561"
server_key_iv: "fa88c941924c8bdcd788361e2cd8cab5"
server_key_version: "encryptionkey@0"
I20260619 06:45:01.364491 22799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:01.377060 22799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:01.381762 22799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:01.381986 22799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:01.400441 22799 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:41937
I20260619 06:45:01.400439 22865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:41937 every 8 connection(s)
I20260619 06:45:01.401871 22799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:45:01.405589 22866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:01.411001 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 22799
I20260619 06:45:01.411213 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:45:01.411855 16701 external_mini_cluster.cc:1428] Setting key 67500277b15a5578ebfd7f0fd0b0cf4b
I20260619 06:45:01.416920 22866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11: Bootstrap starting.
I20260619 06:45:01.420917 22866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:01.422195 22866 log.cc:824] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:01.425189 22866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11: No bootstrap required, opened a new log
Jun 19 06:45:01 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM
I20260619 06:45:01.434036 22866 raft_consensus.cc:348] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } }
I20260619 06:45:01.434492 22866 raft_consensus.cc:374] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:01.434649 22866 raft_consensus.cc:729] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eeb9870492ab483c95bd19f6f547aa11, State: Initialized, Role: FOLLOWER
I20260619 06:45:01.435287 22866 consensus_queue.cc:260] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } }
I20260619 06:45:01.435546 22866 raft_consensus.cc:388] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:45:01.435709 22866 raft_consensus.cc:482] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:45:01.435922 22866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:01.438405 22866 raft_consensus.cc:504] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } }
I20260619 06:45:01.438771 22869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:01.413889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43440 (local address 127.16.79.126:41937)
0619 06:45:01.414741 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:01.414756 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:01.414803 (+ 47us) server_negotiation.cc:407] Connection header received
0619 06:45:01.417143 (+ 2340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:01.417171 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:01.417673 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:01.418083 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:01.420478 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:01.422496 (+ 2018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:01.424549 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:01.425584 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:01.425776 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:01.430480 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:01.430496 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:01.430514 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:01.430562 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:01.434433 (+ 3871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:01.435173 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:01.435187 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:01.435194 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:01.435360 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:01.435772 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:01.435776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:01.435778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:01.436476 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:01.436845 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:01.437206 (+ 361us) server_negotiation.cc:299] Negotiation successful
0619 06:45:01.437603 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":378,"threads_started":1}
I20260619 06:45:01.438997 22866 leader_election.cc:302] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eeb9870492ab483c95bd19f6f547aa11; no voters:
I20260619 06:45:01.439671 22866 leader_election.cc:288] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:45:01.439771 22871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:45:01.442539 22871 raft_consensus.cc:686] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [term 1 LEADER]: Becoming Leader. State: Replica: eeb9870492ab483c95bd19f6f547aa11, State: Running, Role: LEADER
I20260619 06:45:01.443080 22871 consensus_queue.cc:237] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } }
I20260619 06:45:01.443714 22866 sys_catalog.cc:564] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:45:01.445817 22873 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eeb9870492ab483c95bd19f6f547aa11. Latest consensus state: current_term: 1 leader_uuid: "eeb9870492ab483c95bd19f6f547aa11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } }
I20260619 06:45:01.446210 22873 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:01.446583 22872 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eeb9870492ab483c95bd19f6f547aa11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } }
I20260619 06:45:01.446884 22872 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:01.451193 22878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:45:01.455132 22878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:45:01.463406 22878 catalog_manager.cc:1347] Generated new cluster ID: ba64cf8e50a9492cbc84bc1acc9f2cd9
I20260619 06:45:01.463508 22878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:45:01.512930 22878 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:45:01.514807 22878 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:45:01.525928 22878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11: Generated new TSK 0
I20260619 06:45:01.527011 22878 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.65.keytab.
Entry for principal oryx/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.65.keytab.
I20260619 06:45:01.580188 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:41937
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.65.keytab
--principal=oryx/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:01.921437 22892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:01.921752 22892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:01.921918 22892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:01.929489 22892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:01.929661 22892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:01.929832 22892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:45:01.939881 22892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.65.keytab
--principal=oryx/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:41937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:01.941391 22892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:01.942713 22892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:01.956099 22897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:01.956207 22898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:01.956614 22900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:01.957072 22892 server_base.cc:1034] running on GCE node
I20260619 06:45:01.957806 22892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:01.958832 22892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:01.960090 22892 hybrid_clock.cc:648] HybridClock initialized: now 1781851501960031 us; error 51 us; skew 500 ppm
I20260619 06:45:01.960490 22892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:01 dist-test-slave-5w2x krb5kdc[22786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851501, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:01.963876 22892 init.cc:377] Logged in from keytab as oryx/127.16.79.65@KRBTEST.COM (short username oryx)
I20260619 06:45:01.965371 22892 webserver.cc:460] Webserver started at http://127.16.79.65:35055/ using document root <none> and password file <none>
I20260619 06:45:01.966069 22892 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:01.966220 22892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:01.966527 22892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:01.969717 22892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "c48bb43759b34111970076634ed4b417"
format_stamp: "Formatted at 2026-06-19 06:45:01 on dist-test-slave-5w2x"
server_key: "ad028c0b0b1efab6c340acd8f4ded670"
server_key_iv: "9f45d5d8ef6f4ddd75530a11ef01d155"
server_key_version: "encryptionkey@0"
I20260619 06:45:01.970479 22892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "c48bb43759b34111970076634ed4b417"
format_stamp: "Formatted at 2026-06-19 06:45:01 on dist-test-slave-5w2x"
server_key: "ad028c0b0b1efab6c340acd8f4ded670"
server_key_iv: "9f45d5d8ef6f4ddd75530a11ef01d155"
server_key_version: "encryptionkey@0"
I20260619 06:45:01.975699 22892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:45:01.979215 22907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:01.980552 22892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 06:45:01.980731 22892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "c48bb43759b34111970076634ed4b417"
format_stamp: "Formatted at 2026-06-19 06:45:01 on dist-test-slave-5w2x"
server_key: "ad028c0b0b1efab6c340acd8f4ded670"
server_key_iv: "9f45d5d8ef6f4ddd75530a11ef01d155"
server_key_version: "encryptionkey@0"
I20260619 06:45:01.980997 22892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:01.996009 22892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:01.999733 22892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:01.999963 22892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:02.000842 22892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:02.002188 22892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:02.002259 22892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:02.002341 22892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:02.002388 22892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:02.035104 22892 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:33147
I20260619 06:45:02.035152 23019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:33147 every 8 connection(s)
I20260619 06:45:02.036499 22892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:45:02.042601 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 22892
I20260619 06:45:02.042855 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:45:02.043653 16701 external_mini_cluster.cc:1428] Setting key 8728a6212134d09ce96a86f2def4fc5a
Jun 19 06:45:02 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851501, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM
I20260619 06:45:02.058516 23022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:02.038993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:52445 (local address 127.16.79.126:41937)
0619 06:45:02.039435 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:02.039440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:02.040373 (+ 933us) server_negotiation.cc:407] Connection header received
0619 06:45:02.042149 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:02.042154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:02.042247 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:02.042361 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:02.044045 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:02.045021 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:02.047609 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:02.047991 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:02.048093 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:02.052503 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:02.052508 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:02.052512 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:02.052543 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:02.055501 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:02.056261 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:02.056267 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:02.056271 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:02.056400 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:02.056836 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:02.056841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:02.056844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:02.057105 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:02.057258 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:02.057934 (+ 676us) server_negotiation.cc:299] Negotiation successful
0619 06:45:02.058150 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":254,"threads_started":1}
I20260619 06:45:02.059552 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:02.039420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.65:52445)
0619 06:45:02.040215 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:02.040264 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:02.041876 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:02.042596 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:02.042605 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:02.043213 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:02.043876 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:02.043905 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:02.045198 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:02.045202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:02.047447 (+ 2245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:02.047467 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:02.048249 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:02.048260 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:02.048631 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:02.049401 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:02.049439 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:02.052274 (+ 2835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:02.055658 (+ 3384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:02.055667 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:02.055684 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:02.056109 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:02.056514 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:02.056520 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:02.056524 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:02.056716 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:02.057232 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:02.057240 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:02.057584 (+ 344us) client_negotiation.cc:769] Sending connection context
0619 06:45:02.057839 (+ 255us) client_negotiation.cc:240] Negotiation successful
0619 06:45:02.058386 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":276,"threads_started":1}
I20260619 06:45:02.061264 23020 heartbeater.cc:347] Connected to a master server at 127.16.79.126:41937
I20260619 06:45:02.061606 23020 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:02.062597 23020 heartbeater.cc:511] Master 127.16.79.126:41937 requested a full tablet report, sending...
I20260619 06:45:02.064882 22831 ts_manager.cc:194] Registered new tserver with Master: c48bb43759b34111970076634ed4b417 (127.16.79.65:33147)
WARNING: no policy specified for oryx/127.16.79.66@KRBTEST.COM; defaulting to no policy
I20260619 06:45:02.066332 22831 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:52445
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.79.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.66.keytab.
Entry for principal oryx/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.66.keytab.
I20260619 06:45:02.089574 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:41937
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.66.keytab
--principal=oryx/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:02.406512 23026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:02.406800 23026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:02.406958 23026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:02.414224 23026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:02.414333 23026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:02.414507 23026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:45:02.424080 23026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.66.keytab
--principal=oryx/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:41937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:02.425434 23026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:02.426733 23026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:02.439214 23031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:02.439329 23034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:02.439397 23032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:02.440590 23026 server_base.cc:1034] running on GCE node
I20260619 06:45:02.441249 23026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:02.442183 23026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:02.443410 23026 hybrid_clock.cc:648] HybridClock initialized: now 1781851502443353 us; error 53 us; skew 500 ppm
I20260619 06:45:02.443744 23026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:02 dist-test-slave-5w2x krb5kdc[22786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851502, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:02.446933 23026 init.cc:377] Logged in from keytab as oryx/127.16.79.66@KRBTEST.COM (short username oryx)
I20260619 06:45:02.448491 23026 webserver.cc:460] Webserver started at http://127.16.79.66:37535/ using document root <none> and password file <none>
I20260619 06:45:02.449205 23026 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:02.449349 23026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:02.449649 23026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:02.452610 23026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "db74f113a17d4d01959c109f47c628b1"
format_stamp: "Formatted at 2026-06-19 06:45:02 on dist-test-slave-5w2x"
server_key: "ba3311492a8cd22ecff402a16b7a222e"
server_key_iv: "dc7a8d03df3769271e1472936adc636b"
server_key_version: "encryptionkey@0"
I20260619 06:45:02.453467 23026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "db74f113a17d4d01959c109f47c628b1"
format_stamp: "Formatted at 2026-06-19 06:45:02 on dist-test-slave-5w2x"
server_key: "ba3311492a8cd22ecff402a16b7a222e"
server_key_iv: "dc7a8d03df3769271e1472936adc636b"
server_key_version: "encryptionkey@0"
I20260619 06:45:02.458488 23026 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:45:02.461828 23041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:02.463181 23026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:45:02.463383 23026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "db74f113a17d4d01959c109f47c628b1"
format_stamp: "Formatted at 2026-06-19 06:45:02 on dist-test-slave-5w2x"
server_key: "ba3311492a8cd22ecff402a16b7a222e"
server_key_iv: "dc7a8d03df3769271e1472936adc636b"
server_key_version: "encryptionkey@0"
I20260619 06:45:02.463600 23026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:02.479758 23026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:02.483754 23026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:02.484035 23026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:02.485234 23026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:02.486665 23026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:02.486737 23026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:02.486817 23026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:02.486867 23026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:02.517838 23026 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:33659
I20260619 06:45:02.517874 23153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:33659 every 8 connection(s)
I20260619 06:45:02.519207 23026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:45:02.528509 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 23026
I20260619 06:45:02.528692 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:45:02.529170 16701 external_mini_cluster.cc:1428] Setting key 90193b6300a6f804e5de288b41500804
Jun 19 06:45:02 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851502, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM
I20260619 06:45:02.540482 23022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:02.521936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:50361 (local address 127.16.79.126:41937)
0619 06:45:02.522118 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:02.522122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:02.523237 (+ 1115us) server_negotiation.cc:407] Connection header received
0619 06:45:02.525209 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:02.525215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:02.525310 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:02.525423 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:02.527125 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:02.527831 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:02.529616 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:02.530435 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:02.530580 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:02.534821 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:02.534827 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:02.534830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:02.534861 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:02.537427 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:02.538221 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:02.538225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:02.538228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:02.538344 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:02.538794 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:02.538803 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:02.538805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:02.539056 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:02.539205 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:02.539883 (+ 678us) server_negotiation.cc:299] Negotiation successful
0619 06:45:02.540110 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 06:45:02.541483 23156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:02.522323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.66:50361)
0619 06:45:02.523046 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:02.523109 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:02.524890 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:02.525668 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:02.525677 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:02.526371 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:02.526933 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:02.526951 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:02.528000 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:02.528004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:02.529469 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:02.529482 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:02.530682 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:02.530687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:02.530964 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:02.531726 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:02.531756 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:02.534567 (+ 2811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:02.537574 (+ 3007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:02.537584 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:02.537602 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:02.538088 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:02.538459 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:02.538473 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:02.538477 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:02.538671 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:02.539180 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:02.539188 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:02.539533 (+ 345us) client_negotiation.cc:769] Sending connection context
0619 06:45:02.539795 (+ 262us) client_negotiation.cc:240] Negotiation successful
0619 06:45:02.540369 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":296,"threads_started":1}
I20260619 06:45:02.542968 23154 heartbeater.cc:347] Connected to a master server at 127.16.79.126:41937
I20260619 06:45:02.543273 23154 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:02.544024 23154 heartbeater.cc:511] Master 127.16.79.126:41937 requested a full tablet report, sending...
I20260619 06:45:02.545552 22831 ts_manager.cc:194] Registered new tserver with Master: db74f113a17d4d01959c109f47c628b1 (127.16.79.66:33659)
I20260619 06:45:02.546852 22831 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:50361
WARNING: no policy specified for oryx/127.16.79.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.79.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.67.keytab.
Entry for principal oryx/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.67.keytab.
I20260619 06:45:02.575266 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:41937
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.67.keytab
--principal=oryx/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:02.896406 23159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:02.896695 23159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:02.896790 23159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:02.903972 23159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:02.904100 23159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:02.904294 23159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:45:02.913957 23159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.67.keytab
--principal=oryx/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:41937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:02.915185 23159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:02.916455 23159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:02.929447 23165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:02.929780 23164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:02.929423 23167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:02.930269 23159 server_base.cc:1034] running on GCE node
I20260619 06:45:02.930850 23159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:02.931813 23159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:02.933053 23159 hybrid_clock.cc:648] HybridClock initialized: now 1781851502932981 us; error 100 us; skew 500 ppm
I20260619 06:45:02.933430 23159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:02 dist-test-slave-5w2x krb5kdc[22786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851502, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:02.936568 23159 init.cc:377] Logged in from keytab as oryx/127.16.79.67@KRBTEST.COM (short username oryx)
I20260619 06:45:02.938047 23159 webserver.cc:460] Webserver started at http://127.16.79.67:43323/ using document root <none> and password file <none>
I20260619 06:45:02.938759 23159 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:02.938907 23159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:02.939224 23159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:02.942239 23159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "1461b1bd05864c09bef1c0daae653d48"
format_stamp: "Formatted at 2026-06-19 06:45:02 on dist-test-slave-5w2x"
server_key: "f670554cf8260b7d758afc277a7f436e"
server_key_iv: "670ba631e61ecc325c760a584b90885e"
server_key_version: "encryptionkey@0"
I20260619 06:45:02.943063 23159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "1461b1bd05864c09bef1c0daae653d48"
format_stamp: "Formatted at 2026-06-19 06:45:02 on dist-test-slave-5w2x"
server_key: "f670554cf8260b7d758afc277a7f436e"
server_key_iv: "670ba631e61ecc325c760a584b90885e"
server_key_version: "encryptionkey@0"
I20260619 06:45:02.948804 23159 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260619 06:45:02.952446 23174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:02.953886 23159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:45:02.954075 23159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "1461b1bd05864c09bef1c0daae653d48"
format_stamp: "Formatted at 2026-06-19 06:45:02 on dist-test-slave-5w2x"
server_key: "f670554cf8260b7d758afc277a7f436e"
server_key_iv: "670ba631e61ecc325c760a584b90885e"
server_key_version: "encryptionkey@0"
I20260619 06:45:02.954290 23159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:02.969411 23159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:02.973627 23159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:02.973912 23159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:02.974825 23159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:02.976094 23159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:02.976218 23159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:02.976367 23159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:02.976451 23159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:03.008985 23159 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:35113
I20260619 06:45:03.009028 23286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:35113 every 8 connection(s)
I20260619 06:45:03.010309 23159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:45:03.015174 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 23159
I20260619 06:45:03.015344 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:45:03.015880 16701 external_mini_cluster.cc:1428] Setting key dc5a7f66d20c21575fa0d60d50556944
Jun 19 06:45:03 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851502, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM
I20260619 06:45:03.030602 23022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:03.012821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:50321 (local address 127.16.79.126:41937)
0619 06:45:03.013025 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:03.013031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:03.014182 (+ 1151us) server_negotiation.cc:407] Connection header received
0619 06:45:03.016508 (+ 2326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:03.016514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:03.016604 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:03.016711 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:03.018633 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.019448 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.021226 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.021634 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.021728 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:03.024973 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:03.024978 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:03.024981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:03.025012 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:03.027634 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:03.028309 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:03.028313 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:03.028315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:03.028440 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:03.028800 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:03.028802 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:03.028805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:03.029039 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:03.029163 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:03.030018 (+ 855us) server_negotiation.cc:299] Negotiation successful
0619 06:45:03.030204 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:45:03.031646 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:03.013342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.67:50321)
0619 06:45:03.014014 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:03.014057 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:03.016174 (+ 2117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:03.016970 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:03.016978 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:03.017781 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:03.018431 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:03.018449 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.019598 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.019603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:03.021102 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:03.021113 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.021808 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.021812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:03.021995 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:03.022657 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:03.022677 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:03.024776 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:03.027754 (+ 2978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:03.027760 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:03.027772 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:03.028207 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:03.028527 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:03.028530 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:03.028532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:03.028703 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:03.029175 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:03.029182 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:03.029572 (+ 390us) client_negotiation.cc:769] Sending connection context
0619 06:45:03.029886 (+ 314us) client_negotiation.cc:240] Negotiation successful
0619 06:45:03.030420 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":266,"threads_started":1}
I20260619 06:45:03.032974 23287 heartbeater.cc:347] Connected to a master server at 127.16.79.126:41937
I20260619 06:45:03.033283 23287 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:03.033983 23287 heartbeater.cc:511] Master 127.16.79.126:41937 requested a full tablet report, sending...
I20260619 06:45:03.035442 22831 ts_manager.cc:194] Registered new tserver with Master: 1461b1bd05864c09bef1c0daae653d48 (127.16.79.67:35113)
I20260619 06:45:03.036594 22831 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:50321
I20260619 06:45:03.043309 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.16.79.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.79.125@KRBTEST.COM" created.
I20260619 06:45:03.069474 23020 heartbeater.cc:503] Master 127.16.79.126:41937 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.79.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.125.keytab.
Entry for principal oryx/127.16.79.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.125.keytab.
I20260619 06:45:03.089991 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.125:42515
--webserver_interface=127.16.79.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.125.keytab
--principal=oryx/127.16.79.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:41937,127.16.79.125:42515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:03.423369 23292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:03.423653 23292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:03.423739 23292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:03.431043 23292 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:45:03.431165 23292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:03.431212 23292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:03.431265 23292 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:45:03.431310 23292 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:45:03.441265 23292 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:41937,127.16.79.125:42515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.125.keytab
--principal=oryx/127.16.79.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.125:42515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.79.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
W20260619 06:45:03.441586 23292 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.16.79.126:41937,127.16.79.125:42515'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 06:45:03.442621 23292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:03.443835 23292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:03.456008 23297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:03.455946 23298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:03.455982 23300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:03.457083 23292 server_base.cc:1034] running on GCE node
I20260619 06:45:03.457862 23292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:03.459359 23292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:03.460562 23292 hybrid_clock.cc:648] HybridClock initialized: now 1781851503460509 us; error 39 us; skew 500 ppm
I20260619 06:45:03.460882 23292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:03 dist-test-slave-5w2x krb5kdc[22786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851503, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:03.463876 23292 init.cc:377] Logged in from keytab as oryx/127.16.79.125@KRBTEST.COM (short username oryx)
I20260619 06:45:03.465867 23292 webserver.cc:460] Webserver started at http://127.16.79.125:38805/ using document root <none> and password file <none>
I20260619 06:45:03.466570 23292 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:03.466696 23292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:03.466996 23292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:03.470112 23292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/data/instance:
uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7"
format_stamp: "Formatted at 2026-06-19 06:45:03 on dist-test-slave-5w2x"
server_key: "c1b40807c637042fca68615c2160a94f"
server_key_iv: "84f49686a24532d247d3900293c0186d"
server_key_version: "encryptionkey@0"
I20260619 06:45:03.470865 23292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/wal/instance:
uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7"
format_stamp: "Formatted at 2026-06-19 06:45:03 on dist-test-slave-5w2x"
server_key: "c1b40807c637042fca68615c2160a94f"
server_key_iv: "84f49686a24532d247d3900293c0186d"
server_key_version: "encryptionkey@0"
I20260619 06:45:03.475742 23292 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260619 06:45:03.479127 23307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:03.480549 23292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 06:45:03.480731 23292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/wal
uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7"
format_stamp: "Formatted at 2026-06-19 06:45:03 on dist-test-slave-5w2x"
server_key: "c1b40807c637042fca68615c2160a94f"
server_key_iv: "84f49686a24532d247d3900293c0186d"
server_key_version: "encryptionkey@0"
I20260619 06:45:03.480953 23292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:03.508461 23292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:03.512579 23292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:03.512857 23292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:03.531577 23292 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.125:42515
I20260619 06:45:03.531620 23358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.125:42515 every 8 connection(s)
I20260619 06:45:03.532980 23292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/data/info.pb
I20260619 06:45:03.536670 23359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:03.539949 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 23292
I20260619 06:45:03.544494 23359 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:45:03.549767 23154 heartbeater.cc:503] Master 127.16.79.126:41937 was elected leader, sending a full tablet report...
Jun 19 06:45:03 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851503, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.125@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM
Jun 19 06:45:03 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.79.125@KRBTEST.COM
I20260619 06:45:03.567945 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:03.547544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43456 (local address 127.16.79.126:41937)
0619 06:45:03.548957 (+ 1413us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:03.548962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:03.548978 (+ 16us) server_negotiation.cc:407] Connection header received
0619 06:45:03.549024 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:03.549027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:03.549081 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:03.549168 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:03.550610 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.551651 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.553361 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.553759 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.553851 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:03.561031 (+ 7180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:03.561036 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:03.561039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:03.561069 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:03.564323 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:03.565193 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:03.565199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:03.565202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:03.565339 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:03.565820 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:03.565823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:03.565825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:03.566252 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:03.566358 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:03.567207 (+ 849us) server_negotiation.cc:299] Negotiation successful
0619 06:45:03.567446 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1313,"thread_start_us":1233,"threads_started":1}
I20260619 06:45:03.569252 23362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:03.542285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42362 (local address 127.16.79.125:42515)
0619 06:45:03.543027 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:03.543038 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:03.543077 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:45:03.544892 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:03.544913 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:03.545364 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:03.545739 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:03.547553 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.549630 (+ 2077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.551440 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.552419 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.552584 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:03.561578 (+ 8994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:03.561596 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:03.561609 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:03.561643 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:03.565091 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:03.565904 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:03.565915 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:03.565921 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:03.566045 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:03.566431 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:03.566434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:03.566436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:03.566821 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:03.567121 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:03.567550 (+ 429us) server_negotiation.cc:299] Negotiation successful
0619 06:45:03.567860 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":548,"thread_start_us":334,"threads_started":1}
I20260619 06:45:03.569278 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:03.547475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.0.0.1:43456)
0619 06:45:03.548292 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:03.548341 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:03.548498 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:03.549269 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:03.549275 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:03.550159 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:03.550450 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:03.550459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.551765 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.551770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:03.553186 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:03.553196 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.553940 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.553945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:03.554060 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:03.558024 (+ 3964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:03.558061 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:03.560833 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:03.564456 (+ 3623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:03.564464 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:03.564474 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:03.565056 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:03.565451 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:03.565457 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:03.565461 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:03.565699 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:03.566378 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:03.566388 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:03.566795 (+ 407us) client_negotiation.cc:769] Sending connection context
0619 06:45:03.567087 (+ 292us) client_negotiation.cc:240] Negotiation successful
0619 06:45:03.567722 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"spinlock_wait_cycles":14080,"thread_start_us":295,"threads_started":1}
I20260619 06:45:03.571909 23359 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:45:03 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851503, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.125@KRBTEST.COM for oryx/127.16.79.125@KRBTEST.COM
I20260619 06:45:03.583748 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:03.572660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:42515 (local address 127.0.0.1:42366)
0619 06:45:03.572822 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:03.572837 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:03.572950 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:03.573226 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:03.573230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:03.573577 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:03.573692 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:03.573699 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.574655 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.574660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:03.575483 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:03.575501 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.576164 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.576170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:03.576383 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:03.576868 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:03.576884 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:03.578768 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:03.581422 (+ 2654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:03.581429 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:03.581434 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:03.581774 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:03.582106 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:03.582114 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:03.582117 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:03.582236 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:03.582786 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:03.582791 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:03.582940 (+ 149us) client_negotiation.cc:769] Sending connection context
0619 06:45:03.583074 (+ 134us) client_negotiation.cc:240] Negotiation successful
0619 06:45:03.583354 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":3}
I20260619 06:45:03.583798 23362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:03.572829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42366 (local address 127.16.79.125:42515)
0619 06:45:03.572963 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:03.572967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:03.572985 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:45:03.573071 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:03.573074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:03.573119 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:03.573212 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:03.573836 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.574523 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.575616 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.576003 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.576130 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:03.578939 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:03.578944 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:03.578947 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:03.578972 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:03.581294 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:03.581883 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:03.581891 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:03.581895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:03.581999 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:03.582345 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:03.582349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:03.582353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:03.582677 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:03.582827 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:03.583142 (+ 315us) server_negotiation.cc:299] Negotiation successful
0619 06:45:03.583470 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":193,"server-negotiator.queue_time_us":47}
I20260619 06:45:03.588788 23359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: Bootstrap starting.
I20260619 06:45:03.591743 23359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:03.592813 23359 log.cc:824] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:03.595366 23359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: No bootstrap required, opened a new log
I20260619 06:45:03.602320 23359 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } }
I20260619 06:45:03.602640 23359 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:03.602758 23359 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a132b0e2d34fc7b4dcb59ab611b6d7, State: Initialized, Role: FOLLOWER
I20260619 06:45:03.603346 23359 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } }
I20260619 06:45:03.604238 23366 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } } }
I20260619 06:45:03.604521 23366 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:45:03.605221 23359 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 06:45:03.613972 23377 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:45:03.614140 23377 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:45:03.615712 23292 master_runner.cc:428] Detected that this master a3a132b0e2d34fc7b4dcb59ab611b6d7 is joining an existing cluster
I20260619 06:45:03.615803 23292 master_runner.cc:432] Initiating AddMaster RPC to add 127.16.79.125:42515
I20260619 06:45:03.636272 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:03.619738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43470 (local address 127.16.79.126:41937)
0619 06:45:03.619930 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:03.619936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:03.620762 (+ 826us) server_negotiation.cc:407] Connection header received
0619 06:45:03.621057 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:03.621063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:03.621148 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:03.621219 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:03.622848 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.623940 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.626274 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.626905 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.627031 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:03.629841 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:03.629848 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:03.629854 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:03.629895 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:03.633336 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:03.634200 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:03.634207 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:03.634213 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:03.634368 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:03.634748 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:03.634753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:03.634756 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:03.635048 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:03.635245 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:03.635757 (+ 512us) server_negotiation.cc:299] Negotiation successful
0619 06:45:03.635964 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:45:03.636940 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:03.619804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.0.0.1:43470)
0619 06:45:03.620657 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:03.620677 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:03.620875 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:03.621442 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:03.621446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:03.622229 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:03.622652 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:03.622669 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.624166 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.624171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:03.626100 (+ 1929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:03.626120 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.627345 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.627351 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:03.627526 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:03.628263 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:03.628302 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:03.629599 (+ 1297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:03.633503 (+ 3904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:03.633512 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:03.633516 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:03.634060 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:03.634475 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:03.634479 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:03.634481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:03.634617 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:03.635209 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:03.635214 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:03.635426 (+ 212us) client_negotiation.cc:769] Sending connection context
0619 06:45:03.636214 (+ 788us) client_negotiation.cc:240] Negotiation successful
0619 06:45:03.636606 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":38,"spinlock_wait_cycles":3456,"thread_start_us":994,"threads_started":1}
I20260619 06:45:03.637814 23362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:03.620315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42370 (local address 127.16.79.125:42515)
0619 06:45:03.620499 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:03.620505 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:03.621241 (+ 736us) server_negotiation.cc:407] Connection header received
0619 06:45:03.621464 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:03.621470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:03.621527 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:03.621642 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:03.622860 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.623916 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.626403 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.627203 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.627308 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:03.630323 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:03.630331 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:03.630339 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:03.630376 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:03.634044 (+ 3668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:03.635200 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:03.635205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:03.635208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:03.635364 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:03.635810 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:03.635815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:03.635818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:03.636222 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:03.636667 (+ 445us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:03.637245 (+ 578us) server_negotiation.cc:299] Negotiation successful
0619 06:45:03.637485 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:45:03.638800 23383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:03.619960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:42515 (local address 127.0.0.1:42370)
0619 06:45:03.621151 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:03.621168 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:03.621320 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:03.621647 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:03.621651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:03.622335 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:03.622722 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:03.622735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.624189 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.624195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:03.626251 (+ 2056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:03.626271 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.627602 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.627613 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:03.628203 (+ 590us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:03.628881 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:03.628907 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:03.630131 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:03.634525 (+ 4394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:03.634531 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:03.634535 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:03.635082 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:03.635544 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:03.635551 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:03.635556 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:03.635702 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:03.636737 (+ 1035us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:03.636742 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:03.636885 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 06:45:03.638058 (+ 1173us) client_negotiation.cc:240] Negotiation successful
0619 06:45:03.638335 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1043,"mutex_wait_us":82,"spinlock_wait_cycles":7552,"thread_start_us":394,"threads_started":1}
I20260619 06:45:03.641789 23366 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:45:03.642087 23366 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } }
I20260619 06:45:03.643517 23366 leader_election.cc:288] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eeb9870492ab483c95bd19f6f547aa11 (127.16.79.126:41937)
I20260619 06:45:03.645457 22841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eeb9870492ab483c95bd19f6f547aa11" is_pre_election: true
I20260619 06:45:03.646571 23310 leader_election.cc:302] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: a3a132b0e2d34fc7b4dcb59ab611b6d7; no voters: eeb9870492ab483c95bd19f6f547aa11
I20260619 06:45:03.647014 23366 raft_consensus.cc:2726] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Jun 19 06:45:03 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851501, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.126@KRBTEST.COM for oryx/127.16.79.125@KRBTEST.COM
WARNING: no policy specified for oryx/127.16.79.124@KRBTEST.COM; defaulting to no policy
I20260619 06:45:03.659148 23362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:03.643512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42380 (local address 127.16.79.125:42515)
0619 06:45:03.643934 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:03.643939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:03.644908 (+ 969us) server_negotiation.cc:407] Connection header received
0619 06:45:03.645226 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:03.645233 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:03.645347 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:03.645438 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:03.646820 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.647692 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.649148 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.649801 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.649936 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:03.653462 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:03.653467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:03.653470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:03.653501 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:03.656314 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:03.657024 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:03.657029 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:03.657034 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:03.657167 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:03.657551 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:03.657560 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:03.657562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:03.657863 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:03.658010 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:03.658465 (+ 455us) server_negotiation.cc:299] Negotiation successful
0619 06:45:03.658697 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293}
I20260619 06:45:03.659189 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:03.643348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:42515 (local address 127.0.0.1:42380)
0619 06:45:03.644724 (+ 1376us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:03.644793 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:03.645007 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:03.645521 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:03.645529 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:03.646417 (+ 888us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:03.646626 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:03.646644 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.647824 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.647832 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:03.648992 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:03.649005 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.650179 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.650185 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:03.650374 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:03.651061 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:03.651078 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:03.653254 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:03.656453 (+ 3199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:03.656465 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:03.656473 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:03.656895 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:03.657279 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:03.657284 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:03.657288 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:03.657435 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:03.657987 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:03.657996 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:03.658189 (+ 193us) client_negotiation.cc:769] Sending connection context
0619 06:45:03.658370 (+ 181us) client_negotiation.cc:240] Negotiation successful
0619 06:45:03.658698 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1137,"thread_start_us":972,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.79.124@KRBTEST.COM" created.
I20260619 06:45:03.661423 22831 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.16.79.125:42515: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7"
member_type: NON_VOTER
last_known_addr {
host: "127.16.79.125"
port: 42515
}
attrs {
promote: true
}
}
dest_uuid: "eeb9870492ab483c95bd19f6f547aa11"
cas_config_opid_index: -1
I20260619 06:45:03.662617 22831 consensus_queue.cc:237] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: NON_VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: true } }
I20260619 06:45:03.681676 23388 raft_consensus.cc:2932] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a3a132b0e2d34fc7b4dcb59ab611b6d7 (127.16.79.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: NON_VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: true } } }
I20260619 06:45:03.682888 23334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:03.686998 23334 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 FOLLOWER]: Refusing update from remote peer eeb9870492ab483c95bd19f6f547aa11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 06:45:03.687613 23388 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.16.79.125:42515
I20260619 06:45:03.688400 23292 master.cc:537] Master@127.16.79.125:42515 shutting down...
I20260619 06:45:03.689533 23389 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "eeb9870492ab483c95bd19f6f547aa11" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: NON_VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: true } } }
I20260619 06:45:03.689956 23389 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:03.691521 23388 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: NON_VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 06:45:03.695824 22815 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.79.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.124.keytab.
Entry for principal oryx/127.16.79.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.124.keytab.
I20260619 06:45:03.699992 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.124:37437
--webserver_interface=127.16.79.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.124.keytab
--principal=oryx/127.16.79.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:41937,127.16.79.125:42515,127.16.79.124:37437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:03.700083 22815 consensus_peers.cc:487] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 -> Peer a3a132b0e2d34fc7b4dcb59ab611b6d7 (127.16.79.125:42515): Couldn't send request to peer a3a132b0e2d34fc7b4dcb59ab611b6d7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 06:45:03.701122 23292 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:45:03.701562 23292 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:03.701749 23292 tablet_replica.cc:330] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: stopping tablet replica
I20260619 06:45:03.718595 23292 master.cc:552] Master@127.16.79.125:42515 shutdown complete.
I20260619 06:45:03.718698 23292 master_runner.cc:305] Clearing existing system tablet
I20260619 06:45:03.719138 23292 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:03.719238 23292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:03.722270 23292 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 06:45:03.724015 23396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:03.724393 23292 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:03.724689 23292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260619 06:45:03.725132 23292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/wal
uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7"
format_stamp: "Formatted at 2026-06-19 06:45:03 on dist-test-slave-5w2x"
server_key: "c1b40807c637042fca68615c2160a94f"
server_key_iv: "84f49686a24532d247d3900293c0186d"
server_key_version: "encryptionkey@0"
I20260619 06:45:03.730247 23292 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:03.737304 23292 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 06:45:03.737490 23292 log.cc:1196] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 06:45:03.737923 23292 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: Deleting consensus metadata
I20260619 06:45:03.738245 23292 master_runner.cc:315] Copying system tablet from 127.16.79.126:41937
I20260619 06:45:03.740885 23292 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: tablet copy: Beginning tablet copy session from remote peer at address 127.16.79.126:41937
I20260619 06:45:03.759009 23402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:03.741682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.0.0.1:43484)
0619 06:45:03.744200 (+ 2518us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:03.744225 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:03.744397 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:03.744812 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:03.744817 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:03.745411 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:03.745689 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:03.745703 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.747254 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.747264 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:03.748824 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:03.748839 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.749801 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.749808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:03.750003 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:03.750692 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:03.750716 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:03.751789 (+ 1073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:03.756110 (+ 4321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:03.756115 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:03.756121 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:03.756649 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:03.757068 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:03.757072 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:03.757075 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:03.757278 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:03.757907 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:03.757912 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:03.758095 (+ 183us) client_negotiation.cc:769] Sending connection context
0619 06:45:03.758274 (+ 179us) client_negotiation.cc:240] Negotiation successful
0619 06:45:03.758586 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2357,"thread_start_us":2215,"threads_started":1}
I20260619 06:45:03.759071 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:03.741845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43484 (local address 127.16.79.126:41937)
0619 06:45:03.742036 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:03.742045 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:03.744342 (+ 2297us) server_negotiation.cc:407] Connection header received
0619 06:45:03.744555 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:03.744561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:03.744645 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:03.744760 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:03.745859 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.747075 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.748978 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:03.749639 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:03.749772 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:03.752011 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:03.752019 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:03.752027 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:03.752076 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:03.755961 (+ 3885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:03.756789 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:03.756795 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:03.756799 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:03.756947 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:03.757398 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:03.757405 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:03.757411 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:03.757767 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:03.757921 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:03.758456 (+ 535us) server_negotiation.cc:299] Negotiation successful
0619 06:45:03.758701 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 06:45:03.760404 22851 tablet_copy_service.cc:140] P eeb9870492ab483c95bd19f6f547aa11: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a3a132b0e2d34fc7b4dcb59ab611b6d7 ({username='oryx', principal='oryx/127.16.79.125@KRBTEST.COM'} at 127.0.0.1:43484)
I20260619 06:45:03.760684 22851 tablet_copy_service.cc:161] P eeb9870492ab483c95bd19f6f547aa11: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a3a132b0e2d34fc7b4dcb59ab611b6d7 at {username='oryx', principal='oryx/127.16.79.125@KRBTEST.COM'} at 127.0.0.1:43484: session id = a3a132b0e2d34fc7b4dcb59ab611b6d7-00000000000000000000000000000000
I20260619 06:45:03.765180 22851 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 06:45:03.767395 23292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 06:45:03.774283 23292 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: tablet copy: Starting download of 0 data blocks...
I20260619 06:45:03.774693 23292 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: tablet copy: Starting download of 1 WAL segments...
I20260619 06:45:03.777495 23292 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 06:45:03.782015 22851 tablet_copy_service.cc:342] P eeb9870492ab483c95bd19f6f547aa11: Request end of tablet copy session a3a132b0e2d34fc7b4dcb59ab611b6d7-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.16.79.125@KRBTEST.COM'} at 127.0.0.1:43484
I20260619 06:45:03.782213 22851 tablet_copy_service.cc:433] P eeb9870492ab483c95bd19f6f547aa11: ending tablet copy session a3a132b0e2d34fc7b4dcb59ab611b6d7-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a3a132b0e2d34fc7b4dcb59ab611b6d7
I20260619 06:45:03.793236 23292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:03.795744 23403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:03.797098 23404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:03.809978 23406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:03.810204 23292 server_base.cc:1034] running on GCE node
I20260619 06:45:03.810549 23292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:03.812378 23292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:03.813627 23292 hybrid_clock.cc:648] HybridClock initialized: now 1781851503813539 us; error 54 us; skew 500 ppm
I20260619 06:45:03.813915 23292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:03 dist-test-slave-5w2x krb5kdc[22786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851503, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:03.816284 23292 init.cc:377] Logged in from keytab as oryx/127.16.79.125@KRBTEST.COM (short username oryx)
I20260619 06:45:03.819784 23292 webserver.cc:460] Webserver started at http://127.16.79.125:36027/ using document root <none> and password file <none>
I20260619 06:45:03.820119 23292 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:03.820238 23292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:03.823221 23292 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260619 06:45:03.825078 23413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:03.825524 23292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 06:45:03.825662 23292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/wal
uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7"
format_stamp: "Formatted at 2026-06-19 06:45:03 on dist-test-slave-5w2x"
server_key: "c1b40807c637042fca68615c2160a94f"
server_key_iv: "84f49686a24532d247d3900293c0186d"
server_key_version: "encryptionkey@0"
I20260619 06:45:03.825826 23292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:03.850764 23292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:03.854750 23292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:03.879438 23292 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.125:42515
I20260619 06:45:03.879462 23464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.125:42515 every 8 connection(s)
I20260619 06:45:03.880729 23292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-1/data/info.pb
I20260619 06:45:03.885907 23465 sys_catalog.cc:263] Verifying existing consensus state
I20260619 06:45:03.889799 23465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: Bootstrap starting.
I20260619 06:45:03.906150 23465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:45:03.906644 23465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: Bootstrap complete.
I20260619 06:45:03.907604 23465 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: NON_VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: true } }
I20260619 06:45:03.907835 23465 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a3a132b0e2d34fc7b4dcb59ab611b6d7, State: Initialized, Role: LEARNER
I20260619 06:45:03.908041 23465 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: NON_VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: true } }
I20260619 06:45:03.908849 23468 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: NON_VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: true } } }
I20260619 06:45:03.909085 23465 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:45:03.909209 23468 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: This master's current role is: LEARNER
I20260619 06:45:03.919755 23479 catalog_manager.cc:1259] Loaded cluster ID: ba64cf8e50a9492cbc84bc1acc9f2cd9
I20260619 06:45:03.919874 23479 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: loading cluster ID for follower catalog manager: success
I20260619 06:45:03.923237 23479 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: acquiring CA information for follower catalog manager: success
I20260619 06:45:03.925832 23479 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 06:45:04.039000 23287 heartbeater.cc:503] Master 127.16.79.126:41937 was elected leader, sending a full tablet report...
W20260619 06:45:04.080549 23394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:04.080822 23394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:04.080910 23394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:04.088457 23394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:45:04.088562 23394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:04.088606 23394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:04.088670 23394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:45:04.088716 23394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:45:04.098985 23394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:41937,127.16.79.125:42515,127.16.79.124:37437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.124.keytab
--principal=oryx/127.16.79.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.124:37437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.79.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:04.100299 23394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:04.101601 23394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:04.113595 23484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:04.113667 23485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:04.113626 23487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:04.114589 23394 server_base.cc:1034] running on GCE node
I20260619 06:45:04.115347 23394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:04.116787 23394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:04.117971 23394 hybrid_clock.cc:648] HybridClock initialized: now 1781851504117910 us; error 54 us; skew 500 ppm
I20260619 06:45:04.118257 23394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:04 dist-test-slave-5w2x krb5kdc[22786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851504, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:04.121425 23394 init.cc:377] Logged in from keytab as oryx/127.16.79.124@KRBTEST.COM (short username oryx)
I20260619 06:45:04.122877 23394 webserver.cc:460] Webserver started at http://127.16.79.124:41161/ using document root <none> and password file <none>
I20260619 06:45:04.123605 23394 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:04.123734 23394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:04.124054 23394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:04.127524 23394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/data/instance:
uuid: "5e152d5cdd75450982491eddc2eddbcd"
format_stamp: "Formatted at 2026-06-19 06:45:04 on dist-test-slave-5w2x"
server_key: "9276a58458e2d4322e98e8a2253326cb"
server_key_iv: "c08a6369404145200cc1584c39ee897e"
server_key_version: "encryptionkey@0"
I20260619 06:45:04.128458 23394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/wal/instance:
uuid: "5e152d5cdd75450982491eddc2eddbcd"
format_stamp: "Formatted at 2026-06-19 06:45:04 on dist-test-slave-5w2x"
server_key: "9276a58458e2d4322e98e8a2253326cb"
server_key_iv: "c08a6369404145200cc1584c39ee897e"
server_key_version: "encryptionkey@0"
I20260619 06:45:04.133548 23394 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:45:04.136958 23494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:04.138300 23394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:45:04.138480 23394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/wal
uuid: "5e152d5cdd75450982491eddc2eddbcd"
format_stamp: "Formatted at 2026-06-19 06:45:04 on dist-test-slave-5w2x"
server_key: "9276a58458e2d4322e98e8a2253326cb"
server_key_iv: "c08a6369404145200cc1584c39ee897e"
server_key_version: "encryptionkey@0"
I20260619 06:45:04.138654 23394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:04.155879 23394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:04.160504 23394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:04.160737 23394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:04.178670 23394 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.124:37437
I20260619 06:45:04.178689 23545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.124:37437 every 8 connection(s)
I20260619 06:45:04.180095 23394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/data/info.pb
I20260619 06:45:04.182430 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 23394
I20260619 06:45:04.184656 23546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:04.193917 23546 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:45:04 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.79.124@KRBTEST.COM
Jun 19 06:45:04 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851504, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.124@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM
I20260619 06:45:04.207855 23549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.184877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57432 (local address 127.16.79.124:37437)
0619 06:45:04.185670 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:04.185687 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:04.185746 (+ 59us) server_negotiation.cc:407] Connection header received
0619 06:45:04.187819 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:04.187848 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:04.188403 (+ 555us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:04.188876 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:04.190207 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.192302 (+ 2095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.193802 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.194527 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.194737 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.199972 (+ 5235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:04.199978 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:04.199988 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:04.200020 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:04.203673 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.204420 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.204428 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.204437 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.204671 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.205067 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.205072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.205074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.205684 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:04.206011 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:04.206430 (+ 419us) server_negotiation.cc:299] Negotiation successful
0619 06:45:04.206860 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":338,"threads_started":1}
I20260619 06:45:04.211639 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.196544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43498 (local address 127.16.79.126:41937)
0619 06:45:04.196706 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:04.196710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:04.197375 (+ 665us) server_negotiation.cc:407] Connection header received
0619 06:45:04.197586 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:04.197592 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:04.197669 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:04.197771 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:04.199117 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.200100 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.202290 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.202671 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.202767 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.206121 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:04.206126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:04.206130 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:04.206163 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:04.208765 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.209557 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.209566 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.209568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.209695 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.210079 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.210084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.210087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.210402 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:04.210663 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:04.211062 (+ 399us) server_negotiation.cc:299] Negotiation successful
0619 06:45:04.211296 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 06:45:04.211702 23550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.196529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.0.0.1:43498)
0619 06:45:04.197238 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:04.197276 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:04.197454 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:04.197835 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:04.197842 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:04.198603 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:04.198949 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.198963 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.200297 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.200302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.202127 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.202142 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.202861 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.202868 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.203052 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.203634 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:04.203656 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:04.205928 (+ 2272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:04.208895 (+ 2967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:04.208907 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:04.208917 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:04.209436 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:04.209807 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:04.209812 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:04.209816 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:04.209960 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:04.210519 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:04.210528 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:04.210717 (+ 189us) client_negotiation.cc:769] Sending connection context
0619 06:45:04.210973 (+ 256us) client_negotiation.cc:240] Negotiation successful
0619 06:45:04.211316 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":373,"threads_started":1}
I20260619 06:45:04.213088 23546 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:45:04 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851504, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.124@KRBTEST.COM for oryx/127.16.79.125@KRBTEST.COM
I20260619 06:45:04.225226 23550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.213786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:42515 (local address 127.0.0.1:42396)
0619 06:45:04.213978 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:04.214003 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:04.214121 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:04.214529 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:04.214532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:04.214826 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:04.214950 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.214958 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.216224 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.216229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.217134 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.217145 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.217813 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.217819 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.217999 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.218487 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:04.218503 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:04.220446 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:04.223174 (+ 2728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:04.223178 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:04.223181 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:04.223545 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:04.223864 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:04.223867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:04.223869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:04.223986 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:04.224514 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:04.224517 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:04.224639 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 06:45:04.224749 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 06:45:04.224960 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 06:45:04.225267 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.213797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42396 (local address 127.16.79.125:42515)
0619 06:45:04.214285 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:04.214290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:04.214305 (+ 15us) server_negotiation.cc:407] Connection header received
0619 06:45:04.214353 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:04.214356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:04.214412 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:04.214498 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:04.215070 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.216047 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.217260 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.217666 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.217800 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.220618 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:04.220622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:04.220626 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:04.220656 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:04.223036 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.223645 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.223649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.223651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.223774 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.224069 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.224072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.224073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.224417 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:04.224536 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:04.224827 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 06:45:04.224990 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":257,"threads_started":1}
I20260619 06:45:04.226444 23546 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:45:04 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851504, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.124@KRBTEST.COM for oryx/127.16.79.124@KRBTEST.COM
I20260619 06:45:04.238225 23550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.227065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:37437 (local address 127.0.0.1:57434)
0619 06:45:04.227300 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:04.227316 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:04.227392 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:04.227684 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:04.227686 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:04.227994 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:04.228224 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.228234 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.229126 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.229129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.229939 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.229949 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.230567 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.230571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.230707 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.231287 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:04.231304 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:04.233276 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:04.235912 (+ 2636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:04.235915 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:04.235918 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:04.236379 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:04.236723 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:04.236727 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:04.236730 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:04.236883 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:04.237485 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:04.237488 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:04.237613 (+ 125us) client_negotiation.cc:769] Sending connection context
0619 06:45:04.237717 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 06:45:04.237905 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260619 06:45:04.238315 23549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.227158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57434 (local address 127.16.79.124:37437)
0619 06:45:04.227322 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:04.227327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:04.227347 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:45:04.227507 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:04.227512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:04.227577 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:04.227660 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:04.228336 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.229008 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.230055 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.230435 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.230527 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.233452 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:04.233456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:04.233459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:04.233487 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:04.235787 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.236477 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.236481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.236484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.236598 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.237027 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.237031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.237034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.237387 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:04.237493 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:04.237808 (+ 315us) server_negotiation.cc:299] Negotiation successful
0619 06:45:04.238023 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":48}
I20260619 06:45:04.242942 23546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: Bootstrap starting.
I20260619 06:45:04.246218 23546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:04.247218 23546 log.cc:824] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:04.249871 23546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: No bootstrap required, opened a new log
I20260619 06:45:04.256680 23546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } }
I20260619 06:45:04.256968 23546 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:04.257098 23546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e152d5cdd75450982491eddc2eddbcd, State: Initialized, Role: FOLLOWER
I20260619 06:45:04.257727 23546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } }
I20260619 06:45:04.258666 23553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } } }
I20260619 06:45:04.259127 23553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:45:04.259893 23546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [sys.catalog]: configured and running, proceeding with master startup.
W20260619 06:45:04.273880 23564 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:45:04.274026 23564 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:45:04.279012 23394 master_runner.cc:428] Detected that this master 5e152d5cdd75450982491eddc2eddbcd is joining an existing cluster
I20260619 06:45:04.279137 23394 master_runner.cc:432] Initiating AddMaster RPC to add 127.16.79.124:37437
I20260619 06:45:04.301612 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.282925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42412 (local address 127.16.79.125:42515)
0619 06:45:04.283105 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:04.283114 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:04.284572 (+ 1458us) server_negotiation.cc:407] Connection header received
0619 06:45:04.284817 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:04.284823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:04.284924 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:04.285040 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:04.286582 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.287570 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.289646 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.290266 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.290357 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.293627 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:04.293645 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:04.293649 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:04.293686 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:04.297410 (+ 3724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.298519 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.298528 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.298534 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.298730 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.299314 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.299322 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.299329 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.299795 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:04.299958 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:04.300778 (+ 820us) server_negotiation.cc:299] Negotiation successful
0619 06:45:04.301098 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 06:45:04.303073 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.283046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:42515 (local address 127.0.0.1:42412)
0619 06:45:04.284490 (+ 1444us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:04.284509 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:04.284666 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:04.285315 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:04.285319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:04.286086 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:04.286416 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.286428 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.287778 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.287785 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.289493 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.289504 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.290720 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.290730 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.290945 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.291588 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:04.291619 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:04.293438 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:04.297571 (+ 4133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:04.297576 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:04.297581 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:04.298355 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:04.298903 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:04.298915 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:04.298921 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:04.299157 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:04.300057 (+ 900us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:04.300069 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:04.300394 (+ 325us) client_negotiation.cc:769] Sending connection context
0619 06:45:04.301839 (+ 1445us) client_negotiation.cc:240] Negotiation successful
0619 06:45:04.302293 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1306,"mutex_wait_us":1,"spinlock_wait_cycles":21888,"thread_start_us":1114,"threads_started":1}
I20260619 06:45:04.307861 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.283445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43500 (local address 127.16.79.126:41937)
0619 06:45:04.283619 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:04.283626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:04.283694 (+ 68us) server_negotiation.cc:407] Connection header received
0619 06:45:04.284114 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:04.284122 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:04.284239 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:04.284334 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:04.286037 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.287222 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.289367 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.290016 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.290146 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.293627 (+ 3481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:04.293633 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:04.293638 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:04.293677 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:04.297662 (+ 3985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.304498 (+ 6836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.304507 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.304513 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.304760 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.305383 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.305391 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.305397 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.305897 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:04.306069 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:04.306968 (+ 899us) server_negotiation.cc:299] Negotiation successful
0619 06:45:04.307322 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:45:04.309285 23549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.284583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57440 (local address 127.16.79.124:37437)
0619 06:45:04.284753 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:04.284758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:04.291554 (+ 6796us) server_negotiation.cc:407] Connection header received
0619 06:45:04.291879 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:04.291884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:04.291956 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:04.292024 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:04.293725 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.294834 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.296806 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.297478 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.297572 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.299959 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:04.299965 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:04.299970 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:04.300002 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:04.304187 (+ 4185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.306243 (+ 2056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.306248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.306252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.306404 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.306795 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.306800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.306804 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.307194 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:04.307350 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:04.307823 (+ 473us) server_negotiation.cc:299] Negotiation successful
0619 06:45:04.308258 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 06:45:04.309865 23572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.284859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:37437 (local address 127.0.0.1:57440)
0619 06:45:04.291452 (+ 6593us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:04.291478 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:04.291751 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:04.292406 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:04.292413 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:04.293194 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:04.293564 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.293577 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.295044 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.295050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.296617 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.296632 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.297788 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.297795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.297975 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.298582 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:04.298604 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:04.299753 (+ 1149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:04.305585 (+ 5832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:04.305590 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:04.305594 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:04.306117 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:04.306536 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:04.306540 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:04.306544 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:04.306688 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:04.307412 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:04.307417 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:04.307587 (+ 170us) client_negotiation.cc:769] Sending connection context
0619 06:45:04.308702 (+ 1115us) client_negotiation.cc:240] Negotiation successful
0619 06:45:04.309004 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6435,"thread_start_us":6704,"threads_started":1}
I20260619 06:45:04.310173 23570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.282948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.0.0.1:43500)
0619 06:45:04.283597 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:04.283621 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:04.283918 (+ 297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:04.284585 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:04.284590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:04.285326 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:04.285867 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.285880 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.287394 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.287398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.289194 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.289211 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.290442 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.290448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.290768 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.291683 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:04.291707 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:04.293391 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:04.303608 (+ 10217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:04.303611 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:04.303614 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:04.304366 (+ 752us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:04.304928 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:04.304935 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:04.304941 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:04.305182 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:04.306168 (+ 986us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:04.306176 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:04.306473 (+ 297us) client_negotiation.cc:769] Sending connection context
0619 06:45:04.308929 (+ 2456us) client_negotiation.cc:240] Negotiation successful
0619 06:45:04.309400 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":26,"thread_start_us":671,"threads_started":1}
I20260619 06:45:04.314009 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 22892
Jun 19 06:45:04 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.79.65@KRBTEST.COM
I20260619 06:45:04.333243 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.316342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42414 (local address 127.16.79.125:42515)
0619 06:45:04.316523 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:04.316529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:04.324950 (+ 8421us) server_negotiation.cc:407] Connection header received
0619 06:45:04.325273 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:04.325283 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:04.325377 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:04.325484 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:04.326663 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.327856 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.330489 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.331823 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.331943 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.332208 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:04.332810 (+ 602us) server_negotiation.cc:299] Negotiation successful
0619 06:45:04.332950 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:45:04.334007 23575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.316241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:42515 (local address 127.0.0.1:42414)
0619 06:45:04.324844 (+ 8603us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:04.324867 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:04.325086 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:04.325734 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:04.325742 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:04.325776 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:04.326423 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.326442 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.328092 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.328099 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.330311 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.330325 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.332373 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.332379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.332578 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.332655 (+ 77us) client_negotiation.cc:769] Sending connection context
0619 06:45:04.333484 (+ 829us) client_negotiation.cc:240] Negotiation successful
0619 06:45:04.333618 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8226,"thread_start_us":8076,"threads_started":1}
I20260619 06:45:04.334159 23574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.315331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58424 (local address 127.16.79.65:33147)
0619 06:45:04.315937 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:04.315943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:04.315967 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:45:04.316019 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:04.316025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:04.316340 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:04.317674 (+ 1334us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:04.317872 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.318901 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.320287 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.320854 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.321003 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.325952 (+ 4949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:04.325970 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:04.325992 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:04.326036 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:04.329922 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.330812 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.330821 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.330824 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.330979 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.331434 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.331439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.331443 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.331866 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:04.333318 (+ 1452us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:04.333484 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 06:45:04.333746 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":303,"threads_started":1}
I20260619 06:45:04.335623 23015 generic_service.cc:196] Checking for leaks (request via RPC)
Jun 19 06:45:04 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851501, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.126@KRBTEST.COM for oryx/127.16.79.124@KRBTEST.COM
I20260619 06:45:04.345942 23440 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260619 06:45:04.346776 23468 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eeb9870492ab483c95bd19f6f547aa11. Latest consensus state: current_term: 1 leader_uuid: "eeb9870492ab483c95bd19f6f547aa11" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: NON_VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: true } } }
I20260619 06:45:04.347029 23468 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: This master's current role is: LEARNER
I20260619 06:45:04.354744 23575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.325231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:37437 (local address 127.0.0.1:57442)
0619 06:45:04.334258 (+ 9027us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:04.334280 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:04.334461 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:04.335113 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:04.335117 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:04.335584 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:04.335805 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.335820 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.337729 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.337737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.339674 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.339693 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.341316 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.341324 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.341599 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.342418 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:04.342450 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:04.346478 (+ 4028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:04.351763 (+ 5285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:04.351769 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:04.351774 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:04.352372 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:04.352852 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:04.352859 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:04.352863 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:04.353027 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:04.353743 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:04.353748 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:04.353908 (+ 160us) client_negotiation.cc:769] Sending connection context
0619 06:45:04.354057 (+ 149us) client_negotiation.cc:240] Negotiation successful
0619 06:45:04.354341 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8909,"thread_start_us":19465,"threads_started":1}
I20260619 06:45:04.354772 23549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.324805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57442 (local address 127.16.79.124:37437)
0619 06:45:04.325071 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:04.325077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:04.334355 (+ 9278us) server_negotiation.cc:407] Connection header received
0619 06:45:04.334654 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:04.334660 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:04.334795 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:04.334867 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:04.336025 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.337496 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.339926 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.340918 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.341038 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.346743 (+ 5705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:04.346753 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:04.346761 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:04.346812 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:04.351592 (+ 4780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.352535 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.352546 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.352550 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.352703 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.353164 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.353169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.353172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.353526 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:04.353650 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:04.354208 (+ 558us) server_negotiation.cc:299] Negotiation successful
0619 06:45:04.354424 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260619 06:45:04.356223 22831 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.16.79.124:37437: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd"
member_type: NON_VOTER
last_known_addr {
host: "127.16.79.124"
port: 37437
}
attrs {
promote: true
}
}
dest_uuid: "eeb9870492ab483c95bd19f6f547aa11"
cas_config_opid_index: 5
I20260619 06:45:04.357321 22831 consensus_queue.cc:237] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: NON_VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: true } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true } }
I20260619 06:45:04.359105 23573 raft_consensus.cc:2932] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5e152d5cdd75450982491eddc2eddbcd (127.16.79.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: NON_VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: true } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true } } }
I20260619 06:45:04.360954 23440 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 LEARNER]: Refusing update from remote peer eeb9870492ab483c95bd19f6f547aa11: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 06:45:04.363452 23521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:04.367241 23521 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 1 FOLLOWER]: Refusing update from remote peer eeb9870492ab483c95bd19f6f547aa11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 06:45:04.368443 23578 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: NON_VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 06:45:04.369130 23578 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:45:04.369315 23573 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.16.79.124:37437
I20260619 06:45:04.369596 23580 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "eeb9870492ab483c95bd19f6f547aa11" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: NON_VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: true } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true } } }
I20260619 06:45:04.370003 23580 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:04.382504 23394 master.cc:537] Master@127.16.79.124:37437 shutting down...
I20260619 06:45:04.392995 23440 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5e152d5cdd75450982491eddc2eddbcd (127.16.79.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: NON_VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: true } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true } } }
I20260619 06:45:04.395946 23468 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eeb9870492ab483c95bd19f6f547aa11" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: NON_VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: true } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true } } }
I20260619 06:45:04.396322 23468 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: This master's current role is: LEARNER
I20260619 06:45:04.397353 23579 raft_consensus.cc:1050] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11: attempting to promote NON_VOTER a3a132b0e2d34fc7b4dcb59ab611b6d7 to VOTER
I20260619 06:45:04.398028 23579 consensus_queue.cc:237] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true } }
I20260619 06:45:04.408560 23440 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 LEARNER]: Refusing update from remote peer eeb9870492ab483c95bd19f6f547aa11: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 06:45:04.409293 23578 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 06:45:04.412102 23579 raft_consensus.cc:2932] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a3a132b0e2d34fc7b4dcb59ab611b6d7 (127.16.79.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true } } }
I20260619 06:45:04.415859 23578 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "eeb9870492ab483c95bd19f6f547aa11" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true } } }
I20260619 06:45:04.416565 23578 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: This master's current role is: LEADER
W20260619 06:45:04.417094 23500 tablet.cc:2369] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:45:04.417472 23439 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a3a132b0e2d34fc7b4dcb59ab611b6d7 (127.16.79.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true } } }
I20260619 06:45:04.415894 23579 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eeb9870492ab483c95bd19f6f547aa11" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true } } }
I20260619 06:45:04.418915 23579 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: This master's current role is: LEADER
W20260619 06:45:04.421187 22815 consensus_peers.cc:487] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 -> Peer 5e152d5cdd75450982491eddc2eddbcd (127.16.79.124:37437): Couldn't send request to peer 5e152d5cdd75450982491eddc2eddbcd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 06:45:04.421684 23394 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:45:04.421828 23468 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eeb9870492ab483c95bd19f6f547aa11" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true } } }
I20260619 06:45:04.422066 23468 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:45:04.422072 23394 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:04.422264 23394 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: stopping tablet replica
I20260619 06:45:04.428399 23394 master.cc:552] Master@127.16.79.124:37437 shutdown complete.
I20260619 06:45:04.428467 23394 master_runner.cc:305] Clearing existing system tablet
I20260619 06:45:04.428802 23394 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:04.428921 23394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:04.431036 23394 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 06:45:04.432428 23585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:04.432842 23394 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:04.433089 23394 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260619 06:45:04.433194 23394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/wal
uuid: "5e152d5cdd75450982491eddc2eddbcd"
format_stamp: "Formatted at 2026-06-19 06:45:04 on dist-test-slave-5w2x"
server_key: "9276a58458e2d4322e98e8a2253326cb"
server_key_iv: "c08a6369404145200cc1584c39ee897e"
server_key_version: "encryptionkey@0"
I20260619 06:45:04.436473 23394 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:04.440991 23394 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 06:45:04.441159 23394 log.cc:1196] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 06:45:04.441546 23394 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: Deleting consensus metadata
I20260619 06:45:04.441788 23394 master_runner.cc:315] Copying system tablet from 127.16.79.126:41937
I20260619 06:45:04.447829 23394 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: tablet copy: Beginning tablet copy session from remote peer at address 127.16.79.126:41937
I20260619 06:45:04.466724 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.450603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43510 (local address 127.16.79.126:41937)
0619 06:45:04.451015 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:04.451025 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:04.451776 (+ 751us) server_negotiation.cc:407] Connection header received
0619 06:45:04.451871 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:04.451876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:04.451956 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:04.452054 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:04.453687 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.454384 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.456464 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.456929 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.457043 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.459510 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:04.459516 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:04.459522 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:04.459556 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:04.463215 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.464089 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.464096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.464102 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.464316 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:04.464848 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:04.464855 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:04.464861 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:04.465381 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:04.465526 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:04.466222 (+ 696us) server_negotiation.cc:299] Negotiation successful
0619 06:45:04.466461 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260619 06:45:04.467828 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.449081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.0.0.1:43510)
0619 06:45:04.451554 (+ 2473us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:04.451576 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:04.451739 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:04.452247 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:04.452604 (+ 357us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:04.453249 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:04.453508 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.453539 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.454551 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.454557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.456312 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.456326 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.457067 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.457073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.457263 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.458014 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:04.458035 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:04.459309 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:04.463374 (+ 4065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:04.463382 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:04.463389 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:04.463943 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:04.464477 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:04.464487 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:04.464492 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:04.464699 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:04.465642 (+ 943us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:04.465646 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:04.465850 (+ 204us) client_negotiation.cc:769] Sending connection context
0619 06:45:04.467051 (+ 1201us) client_negotiation.cc:240] Negotiation successful
0619 06:45:04.467399 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2335,"thread_start_us":2200,"threads_started":1}
I20260619 06:45:04.468713 22851 tablet_copy_service.cc:140] P eeb9870492ab483c95bd19f6f547aa11: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5e152d5cdd75450982491eddc2eddbcd ({username='oryx', principal='oryx/127.16.79.124@KRBTEST.COM'} at 127.0.0.1:43510)
I20260619 06:45:04.468897 22851 tablet_copy_service.cc:161] P eeb9870492ab483c95bd19f6f547aa11: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5e152d5cdd75450982491eddc2eddbcd at {username='oryx', principal='oryx/127.16.79.124@KRBTEST.COM'} at 127.0.0.1:43510: session id = 5e152d5cdd75450982491eddc2eddbcd-00000000000000000000000000000000
I20260619 06:45:04.471203 22851 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 06:45:04.472945 23394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 06:45:04.477473 23394 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: tablet copy: Starting download of 0 data blocks...
I20260619 06:45:04.477767 23394 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: tablet copy: Starting download of 1 WAL segments...
I20260619 06:45:04.480049 23394 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 06:45:04.483091 22851 tablet_copy_service.cc:342] P eeb9870492ab483c95bd19f6f547aa11: Request end of tablet copy session 5e152d5cdd75450982491eddc2eddbcd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.16.79.124@KRBTEST.COM'} at 127.0.0.1:43510
I20260619 06:45:04.483227 22851 tablet_copy_service.cc:433] P eeb9870492ab483c95bd19f6f547aa11: ending tablet copy session 5e152d5cdd75450982491eddc2eddbcd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5e152d5cdd75450982491eddc2eddbcd
I20260619 06:45:04.493606 23394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:04.497700 23593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:04.497822 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 22892
W20260619 06:45:04.504261 23595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:04.504534 23394 server_base.cc:1034] running on GCE node
W20260619 06:45:04.509250 23592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:04.509933 23394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:04.510603 23394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:04.511874 23394 hybrid_clock.cc:648] HybridClock initialized: now 1781851504511829 us; error 29 us; skew 500 ppm
I20260619 06:45:04.512169 23394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:04 dist-test-slave-5w2x krb5kdc[22786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851504, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:04.514504 23394 init.cc:377] Logged in from keytab as oryx/127.16.79.124@KRBTEST.COM (short username oryx)
I20260619 06:45:04.517189 23394 webserver.cc:460] Webserver started at http://127.16.79.124:37305/ using document root <none> and password file <none>
I20260619 06:45:04.517534 23394 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:04.517644 23394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:04.521070 23394 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260619 06:45:04.522207 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:33147
--local_ip_for_outbound_sockets=127.16.79.65
--tserver_master_addrs=127.16.79.126:41937
--webserver_port=35055
--webserver_interface=127.16.79.65
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.65.keytab
--principal=oryx/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.79.126:41937,127.16.79.125:42515
--tserver_master_addrs=127.16.79.126:41937,127.16.79.125:42515,127.16.79.124:37437 with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:45:04.523243 23602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:04.523687 23394 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260619 06:45:04.523811 23394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/wal
uuid: "5e152d5cdd75450982491eddc2eddbcd"
format_stamp: "Formatted at 2026-06-19 06:45:04 on dist-test-slave-5w2x"
server_key: "9276a58458e2d4322e98e8a2253326cb"
server_key_iv: "c08a6369404145200cc1584c39ee897e"
server_key_version: "encryptionkey@0"
I20260619 06:45:04.524005 23394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:04.537562 23394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:04.542129 23394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:04.578372 23394 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.124:37437
I20260619 06:45:04.578433 23654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.124:37437 every 8 connection(s)
I20260619 06:45:04.579455 23394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/master-2/data/info.pb
I20260619 06:45:04.584780 23655 sys_catalog.cc:263] Verifying existing consensus state
I20260619 06:45:04.588768 23655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: Bootstrap starting.
I20260619 06:45:04.606161 23655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:45:04.606707 23655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: Bootstrap complete.
I20260619 06:45:04.607858 23655 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true } }
I20260619 06:45:04.608179 23655 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5e152d5cdd75450982491eddc2eddbcd, State: Initialized, Role: LEARNER
I20260619 06:45:04.608450 23655 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true } }
I20260619 06:45:04.609684 23655 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:45:04.609448 23658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true } } }
I20260619 06:45:04.609989 23658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [sys.catalog]: This master's current role is: LEARNER
I20260619 06:45:04.620414 23669 catalog_manager.cc:1259] Loaded cluster ID: ba64cf8e50a9492cbc84bc1acc9f2cd9
I20260619 06:45:04.620535 23669 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: loading cluster ID for follower catalog manager: success
I20260619 06:45:04.622969 23669 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: acquiring CA information for follower catalog manager: success
I20260619 06:45:04.625285 23669 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 06:45:04.866083 23671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.859032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54366 (local address 127.16.79.124:37437)
0619 06:45:04.860078 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:04.860083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:04.860109 (+ 26us) server_negotiation.cc:407] Connection header received
0619 06:45:04.860219 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:04.860223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:04.860289 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:04.860385 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:04.860920 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.861888 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.864230 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.865129 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.865228 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.865409 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:04.865702 (+ 293us) server_negotiation.cc:299] Negotiation successful
0619 06:45:04.865814 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":940,"thread_start_us":791,"threads_started":1}
I20260619 06:45:04.866168 23670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:04.858846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:37437 (local address 127.0.0.1:54366)
0619 06:45:04.859361 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:04.859387 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:04.859563 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:04.860450 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:04.860456 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:04.860497 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:04.860751 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.860765 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.862037 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.862044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.864040 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:04.864055 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:04.865304 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:04.865310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:04.865501 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:04.865538 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 06:45:04.865658 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 06:45:04.865801 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":211,"threads_started":1}
I20260619 06:45:04.867233 23630 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260619 06:45:04.867939 23658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader eeb9870492ab483c95bd19f6f547aa11. Latest consensus state: current_term: 1 leader_uuid: "eeb9870492ab483c95bd19f6f547aa11" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true } } }
I20260619 06:45:04.868320 23658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [sys.catalog]: This master's current role is: LEARNER
W20260619 06:45:04.903723 23607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:04.904052 23607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:04.904284 23607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:04.912173 23607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:04.912338 23607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:04.912519 23607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:45:04.922953 23607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.65.keytab
--principal=oryx/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:33147
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=35055
--tserver_master_addrs=127.16.79.126:41937,127.16.79.125:42515,127.16.79.124:37437
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:04.924420 23607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:04.925856 23607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:04.939229 23679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:04.939322 23677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:04.939260 23676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:04.940600 23607 server_base.cc:1034] running on GCE node
I20260619 06:45:04.941239 23607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:04.942219 23607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:04.943444 23607 hybrid_clock.cc:648] HybridClock initialized: now 1781851504943379 us; error 60 us; skew 500 ppm
I20260619 06:45:04.943804 23607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:04 dist-test-slave-5w2x krb5kdc[22786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851504, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:04.947019 23607 init.cc:377] Logged in from keytab as oryx/127.16.79.65@KRBTEST.COM (short username oryx)
I20260619 06:45:04.949105 23607 webserver.cc:460] Webserver started at http://127.16.79.65:35055/ using document root <none> and password file <none>
I20260619 06:45:04.949831 23607 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:04.949991 23607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:04.955463 23607 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260619 06:45:04.958338 23686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:04.959894 23607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 06:45:04.960065 23607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "c48bb43759b34111970076634ed4b417"
format_stamp: "Formatted at 2026-06-19 06:45:01 on dist-test-slave-5w2x"
server_key: "ad028c0b0b1efab6c340acd8f4ded670"
server_key_iv: "9f45d5d8ef6f4ddd75530a11ef01d155"
server_key_version: "encryptionkey@0"
I20260619 06:45:04.961100 23607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:04.980382 23607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:04.984759 23607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:04.985081 23607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:04.986074 23607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:04.987445 23607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:04.987561 23607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:04.987689 23607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:04.987768 23607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:05.022480 23607 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:33147
I20260619 06:45:05.022555 23798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:33147 every 8 connection(s)
I20260619 06:45:05.023969 23607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:45:05.027822 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 23607
I20260619 06:45:05.028239 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 23026
Jun 19 06:45:05 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851504, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM
Jun 19 06:45:05 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851504, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.125@KRBTEST.COM
Jun 19 06:45:05 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.79.66@KRBTEST.COM
I20260619 06:45:05.056929 23803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:05.027021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:45113 (local address 127.16.79.126:41937)
0619 06:45:05.027503 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:05.027510 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:05.028661 (+ 1151us) server_negotiation.cc:407] Connection header received
0619 06:45:05.032332 (+ 3671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:05.032341 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:05.032462 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:05.032566 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:05.035609 (+ 3043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.037200 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.040512 (+ 3312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.041238 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.041338 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:05.047281 (+ 5943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:05.047288 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:05.047293 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:05.047336 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:05.050899 (+ 3563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:05.053293 (+ 2394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:05.053299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:05.053302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:05.053483 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:05.054754 (+ 1271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:05.054759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:05.054762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:05.055125 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:05.055282 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:05.056287 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0619 06:45:05.056520 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":241,"threads_started":1}
I20260619 06:45:05.059458 23805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:05.027688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.65:45113)
0619 06:45:05.028492 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:05.028543 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:05.031875 (+ 3332us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:05.033302 (+ 1427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:05.033318 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:05.034293 (+ 975us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:05.035336 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:05.035369 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.037467 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.037476 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:05.040288 (+ 2812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:05.040309 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.041699 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.041706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:05.042041 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:05.043084 (+ 1043us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:05.043124 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:05.046993 (+ 3869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:05.051118 (+ 4125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:05.051128 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:05.051147 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:05.053135 (+ 1988us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:05.054387 (+ 1252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:05.054392 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:05.054396 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:05.054630 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:05.055384 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:05.055393 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:05.055828 (+ 435us) client_negotiation.cc:769] Sending connection context
0619 06:45:05.057208 (+ 1380us) client_negotiation.cc:240] Negotiation successful
0619 06:45:05.057872 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":432,"threads_started":1}
I20260619 06:45:05.061275 23800 heartbeater.cc:347] Connected to a master server at 127.16.79.126:41937
Jun 19 06:45:05 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851504, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.124@KRBTEST.COM
I20260619 06:45:05.061609 23800 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:05.062022 23804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:05.027158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:60655 (local address 127.16.79.125:42515)
0619 06:45:05.028998 (+ 1840us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:05.029004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:05.042058 (+ 13054us) server_negotiation.cc:407] Connection header received
0619 06:45:05.042137 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:05.042142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:05.042232 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:05.042330 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:05.043852 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.045893 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.049101 (+ 3208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.049997 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.050120 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:05.054466 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:05.054475 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:05.054483 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:05.054533 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:05.058347 (+ 3814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:05.059218 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:05.059226 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:05.059230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:05.059369 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:05.059792 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:05.059798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:05.059804 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:05.060239 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:05.060389 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:05.061425 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0619 06:45:05.061638 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1724,"thread_start_us":1970,"threads_started":1}
I20260619 06:45:05.062615 23800 heartbeater.cc:511] Master 127.16.79.126:41937 requested a full tablet report, sending...
I20260619 06:45:05.062960 23806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:05.028193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:42515 (local address 127.16.79.65:60655)
0619 06:45:05.041806 (+ 13613us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:05.041826 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:05.041948 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:05.042571 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:05.042576 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:05.043294 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:05.043646 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:05.043661 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.046104 (+ 2443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.046109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:05.048892 (+ 2783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:05.048912 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.050504 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.050513 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:05.050770 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:05.051380 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:05.051401 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:05.054230 (+ 2829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:05.058470 (+ 4240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:05.058476 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:05.058480 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:05.059073 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:05.059507 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:05.059516 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:05.059522 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:05.059666 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:05.060453 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:05.060458 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:05.060623 (+ 165us) client_negotiation.cc:769] Sending connection context
0619 06:45:05.060778 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 06:45:05.061060 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13487,"mutex_wait_us":709,"thread_start_us":12002,"threads_started":1}
I20260619 06:45:05.064154 23801 heartbeater.cc:347] Connected to a master server at 127.16.79.125:42515
I20260619 06:45:05.064285 23801 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:05.064560 22831 ts_manager.cc:194] Re-registered known tserver with Master: c48bb43759b34111970076634ed4b417 (127.16.79.65:33147)
I20260619 06:45:05.064788 23801 heartbeater.cc:511] Master 127.16.79.125:42515 requested a full tablet report, sending...
I20260619 06:45:05.065676 22831 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:45113
I20260619 06:45:05.066666 23430 ts_manager.cc:194] Registered new tserver with Master: c48bb43759b34111970076634ed4b417 (127.16.79.65:33147)
I20260619 06:45:05.067234 23808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:05.029405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52516 (local address 127.16.79.66:33659)
0619 06:45:05.040889 (+ 11484us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:05.040899 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:05.040933 (+ 34us) server_negotiation.cc:407] Connection header received
0619 06:45:05.041006 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:05.041015 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:05.041282 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:05.044859 (+ 3577us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:05.045192 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.046836 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.049076 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.050002 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.051753 (+ 1751us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:05.056680 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:05.056701 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:05.056723 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:05.056780 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:05.062486 (+ 5706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:05.063623 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:05.063634 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:05.063641 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:05.063835 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:05.064367 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:05.064372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:05.064375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:05.064922 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:05.066261 (+ 1339us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:05.066437 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 06:45:05.066717 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11302,"thread_start_us":11146,"threads_started":1}
I20260619 06:45:05.068782 23149 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:05.069979 23671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:05.041582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:48699 (local address 127.16.79.124:37437)
0619 06:45:05.041764 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:05.041769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:05.052107 (+ 10338us) server_negotiation.cc:407] Connection header received
0619 06:45:05.052281 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:05.052290 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:05.052386 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:05.052493 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:05.053749 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.054803 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.056697 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.057310 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.057400 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:05.062605 (+ 5205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:05.062614 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:05.062620 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:05.062664 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:05.066207 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:05.067184 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:05.067190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:05.067193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:05.067340 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:05.068173 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:05.068179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:05.068181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:05.068514 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:05.068694 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:05.069129 (+ 435us) server_negotiation.cc:299] Negotiation successful
0619 06:45:05.069451 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 06:45:05.070250 23809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:05.041910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:37437 (local address 127.16.79.65:48699)
0619 06:45:05.051970 (+ 10060us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:05.051988 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:05.052119 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:05.052548 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:05.052554 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:05.053188 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:05.053504 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:05.053517 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.054967 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.054972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:05.056530 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:05.056542 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.057838 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.057845 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:05.058010 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:05.059263 (+ 1253us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:05.059282 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:05.062363 (+ 3081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:05.066400 (+ 4037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:05.066404 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:05.066409 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:05.067019 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:05.067744 (+ 725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:05.067749 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:05.067752 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:05.067922 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:05.068661 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:05.068666 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:05.068820 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 06:45:05.068966 (+ 146us) client_negotiation.cc:240] Negotiation successful
0619 06:45:05.069342 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9923,"mutex_wait_us":1,"thread_start_us":9256,"threads_started":1}
I20260619 06:45:05.071456 23799 heartbeater.cc:347] Connected to a master server at 127.16.79.124:37437
I20260619 06:45:05.071589 23799 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:05.071930 23799 heartbeater.cc:511] Master 127.16.79.124:37437 requested a full tablet report, sending...
I20260619 06:45:05.073534 23620 ts_manager.cc:194] Registered new tserver with Master: c48bb43759b34111970076634ed4b417 (127.16.79.65:33147)
I20260619 06:45:05.154742 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 23026
I20260619 06:45:05.171304 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:33659
--local_ip_for_outbound_sockets=127.16.79.66
--tserver_master_addrs=127.16.79.126:41937
--webserver_port=37535
--webserver_interface=127.16.79.66
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.66.keytab
--principal=oryx/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.79.126:41937,127.16.79.125:42515
--tserver_master_addrs=127.16.79.126:41937,127.16.79.125:42515,127.16.79.124:37437 with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:45:05.389075 23579 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eeb9870492ab483c95bd19f6f547aa11" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: NON_VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: true } } }
I20260619 06:45:05.389461 23579 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:05.390626 23579 raft_consensus.cc:1050] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11: attempting to promote NON_VOTER 5e152d5cdd75450982491eddc2eddbcd to VOTER
I20260619 06:45:05.391283 23579 consensus_queue.cc:237] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } }
I20260619 06:45:05.393292 23439 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 FOLLOWER]: Refusing update from remote peer eeb9870492ab483c95bd19f6f547aa11: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260619 06:45:05.393301 23630 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 1 LEARNER]: Refusing update from remote peer eeb9870492ab483c95bd19f6f547aa11: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260619 06:45:05.393891 23813 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260619 06:45:05.394300 23812 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260619 06:45:05.397038 23813 raft_consensus.cc:2932] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 5e152d5cdd75450982491eddc2eddbcd (127.16.79.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } } }
I20260619 06:45:05.397697 23439 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 5e152d5cdd75450982491eddc2eddbcd (127.16.79.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } } }
I20260619 06:45:05.401356 23813 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "eeb9870492ab483c95bd19f6f547aa11" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } } }
I20260619 06:45:05.401616 23813 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:05.401679 23579 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eeb9870492ab483c95bd19f6f547aa11" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } } }
I20260619 06:45:05.402024 23579 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:05.402714 23630 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 5e152d5cdd75450982491eddc2eddbcd (127.16.79.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } } }
I20260619 06:45:05.408321 23579 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eeb9870492ab483c95bd19f6f547aa11" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } } }
I20260619 06:45:05.408594 23579 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11 [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:05.408782 23816 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eeb9870492ab483c95bd19f6f547aa11" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } } }
I20260619 06:45:05.409312 23816 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:45:05.409880 23817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eeb9870492ab483c95bd19f6f547aa11" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } } }
I20260619 06:45:05.410239 23817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [sys.catalog]: This master's current role is: FOLLOWER
W20260619 06:45:05.513288 23811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:05.513618 23811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:05.513806 23811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:05.521246 23811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:05.521377 23811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:05.521550 23811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:45:05.532505 23811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.66.keytab
--principal=oryx/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:33659
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=37535
--tserver_master_addrs=127.16.79.126:41937,127.16.79.125:42515,127.16.79.124:37437
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:05.533895 23811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:05.535229 23811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:05.549206 23823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:05.549285 23825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:05.549566 23822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:05.549829 23811 server_base.cc:1034] running on GCE node
I20260619 06:45:05.550537 23811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:05.551435 23811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:05.552670 23811 hybrid_clock.cc:648] HybridClock initialized: now 1781851505552597 us; error 59 us; skew 500 ppm
I20260619 06:45:05.552994 23811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:05 dist-test-slave-5w2x krb5kdc[22786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851505, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:05.556051 23811 init.cc:377] Logged in from keytab as oryx/127.16.79.66@KRBTEST.COM (short username oryx)
I20260619 06:45:05.557677 23811 webserver.cc:460] Webserver started at http://127.16.79.66:37535/ using document root <none> and password file <none>
I20260619 06:45:05.558475 23811 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:05.558616 23811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:05.564234 23811 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:45:05.567168 23832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:05.568645 23811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:45:05.568835 23811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "db74f113a17d4d01959c109f47c628b1"
format_stamp: "Formatted at 2026-06-19 06:45:02 on dist-test-slave-5w2x"
server_key: "ba3311492a8cd22ecff402a16b7a222e"
server_key_iv: "dc7a8d03df3769271e1472936adc636b"
server_key_version: "encryptionkey@0"
I20260619 06:45:05.569837 23811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:05.586539 23811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:05.591255 23811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:05.591548 23811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:05.592576 23811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:05.593910 23811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:05.594048 23811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:05.594174 23811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:05.594264 23811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:05.627285 23811 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:33659
I20260619 06:45:05.627408 23944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:33659 every 8 connection(s)
I20260619 06:45:05.628841 23811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:45:05.632171 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 23811
I20260619 06:45:05.632503 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 23159
Jun 19 06:45:05 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851505, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.125@KRBTEST.COM
Jun 19 06:45:05 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851505, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.124@KRBTEST.COM
Jun 19 06:45:05 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.79.67@KRBTEST.COM
Jun 19 06:45:05 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851505, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM
I20260619 06:45:05.670792 23956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:05.645419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:57359 (local address 127.16.79.124:37437)
0619 06:45:05.648538 (+ 3119us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:05.648542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:05.648606 (+ 64us) server_negotiation.cc:407] Connection header received
0619 06:45:05.649767 (+ 1161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:05.649775 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:05.649880 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:05.650009 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:05.652339 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.653512 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.656183 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.656883 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.657013 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:05.663088 (+ 6075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:05.663097 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:05.663102 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:05.663144 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:05.666918 (+ 3774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:05.667861 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:05.667866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:05.667870 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:05.668067 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:05.668626 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:05.668631 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:05.668635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:05.668966 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:05.669101 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:05.670146 (+ 1045us) server_negotiation.cc:299] Negotiation successful
0619 06:45:05.670368 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3002,"thread_start_us":2887,"threads_started":1}
I20260619 06:45:05.672034 23949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:05.631826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:34163 (local address 127.16.79.125:42515)
0619 06:45:05.652290 (+ 20464us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:05.652299 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:05.652324 (+ 25us) server_negotiation.cc:407] Connection header received
0619 06:45:05.652625 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:05.652633 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:05.652758 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:05.652873 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:05.654279 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.655401 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.657034 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.657650 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.657778 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:05.662825 (+ 5047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:05.662830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:05.662833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:05.662872 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:05.666716 (+ 3844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:05.667861 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:05.667866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:05.667870 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:05.668011 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:05.668626 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:05.668631 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:05.668635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:05.668975 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:05.669106 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:05.671493 (+ 2387us) server_negotiation.cc:299] Negotiation successful
0619 06:45:05.671709 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20285,"thread_start_us":20499,"threads_started":1}
I20260619 06:45:05.672127 23955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:05.645293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:37437 (local address 127.16.79.66:57359)
0619 06:45:05.647319 (+ 2026us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:05.647333 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:05.649314 (+ 1981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:05.650321 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:05.650332 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:05.651249 (+ 917us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:05.652110 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:05.652162 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.653685 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.653690 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:05.655999 (+ 2309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:05.656022 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.657294 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.657301 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:05.657566 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:05.658574 (+ 1008us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:05.658607 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:05.662501 (+ 3894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:05.667064 (+ 4563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:05.667069 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:05.667076 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:05.667720 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:05.668200 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:05.668204 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:05.668208 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:05.668441 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:05.669182 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:05.669191 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:05.669699 (+ 508us) client_negotiation.cc:769] Sending connection context
0619 06:45:05.670015 (+ 316us) client_negotiation.cc:240] Negotiation successful
0619 06:45:05.670598 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1917,"mutex_wait_us":96,"spinlock_wait_cycles":50048,"thread_start_us":1762,"threads_started":1}
I20260619 06:45:05.672983 23954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:05.634271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44838 (local address 127.16.79.67:35113)
0619 06:45:05.648993 (+ 14722us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:05.648999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:05.649663 (+ 664us) server_negotiation.cc:407] Connection header received
0619 06:45:05.649735 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:05.649750 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:05.649933 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:05.650093 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:05.652297 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.653838 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.656701 (+ 2863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.657529 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.657661 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:05.663474 (+ 5813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:05.663489 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:05.663509 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:05.663562 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:05.669777 (+ 6215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:05.670601 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:05.670610 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:05.670614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:05.670772 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:05.671215 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:05.671221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:05.671224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:05.671680 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:05.671972 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:05.672322 (+ 350us) server_negotiation.cc:299] Negotiation successful
0619 06:45:05.672585 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14591,"thread_start_us":14476,"threads_started":1}
I20260619 06:45:05.673542 23951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:05.632475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:42515 (local address 127.16.79.66:34163)
0619 06:45:05.644924 (+ 12449us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:05.645004 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:05.652441 (+ 7437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:05.653122 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:05.653130 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:05.653740 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:05.654111 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:05.654122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.655576 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.655580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:05.656878 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:05.656891 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.658201 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.658211 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:05.658471 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:05.659035 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:05.659057 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:05.662599 (+ 3542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:05.666909 (+ 4310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:05.666919 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:05.666937 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:05.667720 (+ 783us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:05.668169 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:05.668176 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:05.668179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:05.668509 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:05.669182 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:05.669190 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:05.669700 (+ 510us) client_negotiation.cc:769] Sending connection context
0619 06:45:05.670015 (+ 315us) client_negotiation.cc:240] Negotiation successful
0619 06:45:05.673181 (+ 3166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12284,"mutex_wait_us":2705,"spinlock_wait_cycles":71168,"thread_start_us":11959,"threads_started":1}
I20260619 06:45:05.673992 23945 heartbeater.cc:347] Connected to a master server at 127.16.79.124:37437
I20260619 06:45:05.674314 23945 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:05.674333 23282 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:05.674768 23947 heartbeater.cc:347] Connected to a master server at 127.16.79.125:42515
I20260619 06:45:05.674930 23947 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:05.675388 23945 heartbeater.cc:511] Master 127.16.79.124:37437 requested a full tablet report, sending...
I20260619 06:45:05.675504 23947 heartbeater.cc:511] Master 127.16.79.125:42515 requested a full tablet report, sending...
I20260619 06:45:05.677526 23430 ts_manager.cc:194] Registered new tserver with Master: db74f113a17d4d01959c109f47c628b1 (127.16.79.66:33659)
I20260619 06:45:05.678812 23620 ts_manager.cc:194] Registered new tserver with Master: db74f113a17d4d01959c109f47c628b1 (127.16.79.66:33659)
I20260619 06:45:05.679615 23950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:05.632475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.66:48293)
0619 06:45:05.646349 (+ 13874us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:05.646375 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:05.649321 (+ 2946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:05.654573 (+ 5252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:05.654580 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:05.655122 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:05.655455 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:05.655469 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.657965 (+ 2496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.657971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:05.660598 (+ 2627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:05.660613 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.662282 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.662290 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:05.662521 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:05.663094 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:05.663116 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:05.665774 (+ 2658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:05.670430 (+ 4656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:05.670435 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:05.670438 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:05.671256 (+ 818us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:05.672591 (+ 1335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:05.672599 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:05.672603 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:05.672783 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:05.678518 (+ 5735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:05.678526 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:05.678720 (+ 194us) client_negotiation.cc:769] Sending connection context
0619 06:45:05.678857 (+ 137us) client_negotiation.cc:240] Negotiation successful
0619 06:45:05.679214 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11547,"mutex_wait_us":805,"thread_start_us":11303,"threads_started":1}
I20260619 06:45:05.679682 23952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:05.633098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:48293 (local address 127.16.79.126:41937)
0619 06:45:05.654256 (+ 21158us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:05.654263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:05.654281 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:45:05.654345 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:05.654351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:05.654421 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:05.654539 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:05.656224 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.657302 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.661409 (+ 4107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:05.662054 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:05.662185 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:05.666071 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:05.666078 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:05.666082 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:05.666131 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:05.669464 (+ 3333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:05.671387 (+ 1923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:05.671392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:05.671395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:05.671549 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:05.672893 (+ 1344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:05.672898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:05.672901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:05.676732 (+ 3831us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:05.677008 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:05.679022 (+ 2014us) server_negotiation.cc:299] Negotiation successful
0619 06:45:05.679265 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21030,"thread_start_us":20928,"threads_started":1}
I20260619 06:45:05.680541 23946 heartbeater.cc:347] Connected to a master server at 127.16.79.126:41937
I20260619 06:45:05.680671 23946 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:05.681063 23946 heartbeater.cc:511] Master 127.16.79.126:41937 requested a full tablet report, sending...
I20260619 06:45:05.682042 22831 ts_manager.cc:194] Re-registered known tserver with Master: db74f113a17d4d01959c109f47c628b1 (127.16.79.66:33659)
I20260619 06:45:05.683243 22831 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:48293
I20260619 06:45:05.758991 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 23159
I20260619 06:45:05.775135 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:35113
--local_ip_for_outbound_sockets=127.16.79.67
--tserver_master_addrs=127.16.79.126:41937
--webserver_port=43323
--webserver_interface=127.16.79.67
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.67.keytab
--principal=oryx/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.16.79.126:41937,127.16.79.125:42515
--tserver_master_addrs=127.16.79.126:41937,127.16.79.125:42515,127.16.79.124:37437 with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:45:06.068725 23800 heartbeater.cc:503] Master 127.16.79.126:41937 was elected leader, sending a full tablet report...
W20260619 06:45:06.109658 23958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:06.109997 23958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:06.110193 23958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:06.117887 23958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:06.118001 23958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:06.118202 23958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:45:06.128224 23958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.67.keytab
--principal=oryx/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:35113
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=43323
--tserver_master_addrs=127.16.79.126:41937,127.16.79.125:42515,127.16.79.124:37437
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:06.129571 23958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:06.130927 23958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:06.144088 23965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:06.144202 23967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:06.144129 23964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:06.145016 23958 server_base.cc:1034] running on GCE node
I20260619 06:45:06.145552 23958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:06.146463 23958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:06.147657 23958 hybrid_clock.cc:648] HybridClock initialized: now 1781851506147597 us; error 51 us; skew 500 ppm
I20260619 06:45:06.147971 23958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:06 dist-test-slave-5w2x krb5kdc[22786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851506, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:06.151050 23958 init.cc:377] Logged in from keytab as oryx/127.16.79.67@KRBTEST.COM (short username oryx)
I20260619 06:45:06.152453 23958 webserver.cc:460] Webserver started at http://127.16.79.67:43323/ using document root <none> and password file <none>
I20260619 06:45:06.153131 23958 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:06.153291 23958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:06.158387 23958 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260619 06:45:06.161074 23974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:06.162411 23958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 06:45:06.162578 23958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "1461b1bd05864c09bef1c0daae653d48"
format_stamp: "Formatted at 2026-06-19 06:45:02 on dist-test-slave-5w2x"
server_key: "f670554cf8260b7d758afc277a7f436e"
server_key_iv: "670ba631e61ecc325c760a584b90885e"
server_key_version: "encryptionkey@0"
I20260619 06:45:06.163529 23958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:06.183885 23958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:06.188295 23958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:06.188563 23958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:06.189498 23958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:06.190819 23958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:06.190954 23958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:06.191077 23958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:06.191155 23958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:06.223600 23958 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:35113
I20260619 06:45:06.223703 24086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:35113 every 8 connection(s)
I20260619 06:45:06.225237 23958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:45:06.235579 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 23958
Jun 19 06:45:06 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851506, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM
Jun 19 06:45:06 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851506, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.124@KRBTEST.COM
I20260619 06:45:06.263988 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.229361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:44789 (local address 127.16.79.124:37437)
0619 06:45:06.244686 (+ 15325us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:06.244696 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:06.244721 (+ 25us) server_negotiation.cc:407] Connection header received
0619 06:45:06.244785 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:06.244789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:06.244877 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:06.244983 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:06.246021 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.247246 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.250376 (+ 3130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.251050 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.251143 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.254807 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:06.254817 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:06.254825 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:06.254873 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:06.258993 (+ 4120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.260917 (+ 1924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.260922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.260926 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.261064 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.261568 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.261575 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.261581 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.262014 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:06.262170 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:06.263138 (+ 968us) server_negotiation.cc:299] Negotiation successful
0619 06:45:06.263436 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15133,"thread_start_us":15683,"threads_started":1}
I20260619 06:45:06.265275 24093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.229289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:37437 (local address 127.16.79.67:44789)
0619 06:45:06.229899 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:06.229918 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:06.233174 (+ 3256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:06.245023 (+ 11849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:06.245027 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:06.245555 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:06.245860 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:06.245879 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.248039 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.248047 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:06.250204 (+ 2157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:06.250229 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.251229 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.251236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:06.251428 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.252031 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:06.252057 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:06.254547 (+ 2490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:06.259951 (+ 5404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:06.259965 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:06.259990 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:06.260798 (+ 808us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:06.261157 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:06.261163 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:06.261170 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:06.261455 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:06.262170 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:06.262183 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:06.262656 (+ 473us) client_negotiation.cc:769] Sending connection context
0619 06:45:06.262973 (+ 317us) client_negotiation.cc:240] Negotiation successful
0619 06:45:06.263542 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"spinlock_wait_cycles":6528,"thread_start_us":981,"threads_started":1}
I20260619 06:45:06.267100 24087 heartbeater.cc:347] Connected to a master server at 127.16.79.124:37437
I20260619 06:45:06.267488 24087 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:06.268532 24087 heartbeater.cc:511] Master 127.16.79.124:37437 requested a full tablet report, sending...
I20260619 06:45:06.269781 24091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.228090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:55911 (local address 127.16.79.126:41937)
0619 06:45:06.228571 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:06.228578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:06.230003 (+ 1425us) server_negotiation.cc:407] Connection header received
0619 06:45:06.237671 (+ 7668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:06.237681 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:06.237807 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:06.237951 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:06.240937 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.242441 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.246761 (+ 4320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.247596 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.247751 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.255037 (+ 7286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:06.255047 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:06.255055 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:06.255131 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:06.259743 (+ 4612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.266321 (+ 6578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.266330 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.266337 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.267234 (+ 897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.267793 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.267802 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.267807 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.268448 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:06.268585 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:06.269124 (+ 539us) server_negotiation.cc:299] Negotiation successful
0619 06:45:06.269372 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":216,"threads_started":1}
I20260619 06:45:06.270771 23620 ts_manager.cc:194] Registered new tserver with Master: 1461b1bd05864c09bef1c0daae653d48 (127.16.79.67:35113)
I20260619 06:45:06.271078 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.228853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.67:55911)
0619 06:45:06.229795 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:06.229865 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:06.236763 (+ 6898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:06.238325 (+ 1562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:06.238343 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:06.239565 (+ 1222us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:06.240706 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:06.240749 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.243231 (+ 2482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.243238 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:06.246582 (+ 3344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:06.246598 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.248347 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.248354 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:06.248632 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.249426 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:06.249454 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:06.254808 (+ 5354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:06.260979 (+ 6171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:06.260986 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:06.260993 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:06.266147 (+ 5154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:06.267422 (+ 1275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:06.267429 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:06.267436 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:06.267641 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:06.268669 (+ 1028us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:06.268673 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:06.268819 (+ 146us) client_negotiation.cc:769] Sending connection context
0619 06:45:06.270383 (+ 1564us) client_negotiation.cc:240] Negotiation successful
0619 06:45:06.270669 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"spinlock_wait_cycles":31488,"thread_start_us":498,"threads_started":1}
I20260619 06:45:06.272444 24088 heartbeater.cc:347] Connected to a master server at 127.16.79.126:41937
I20260619 06:45:06.272585 24088 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:06.273000 24088 heartbeater.cc:511] Master 127.16.79.126:41937 requested a full tablet report, sending...
I20260619 06:45:06.273977 22831 ts_manager.cc:194] Re-registered known tserver with Master: 1461b1bd05864c09bef1c0daae653d48 (127.16.79.67:35113)
I20260619 06:45:06.274904 22831 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:55911
Jun 19 06:45:06 dist-test-slave-5w2x krb5kdc[22786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851506, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.125@KRBTEST.COM
I20260619 06:45:06.287914 24095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.230718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:42515 (local address 127.16.79.67:52277)
0619 06:45:06.245110 (+ 14392us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:06.245130 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:06.245254 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:06.248875 (+ 3621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:06.248885 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:06.249534 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:06.249951 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:06.249970 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.252305 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.252311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:06.261342 (+ 9031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:06.261361 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.276803 (+ 15442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.276810 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:06.277008 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.277734 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:06.277756 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:06.280080 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:06.284960 (+ 4880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:06.284965 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:06.284968 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:06.285530 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:06.286040 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:06.286046 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:06.286053 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:06.286231 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:06.286856 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:06.286862 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:06.287025 (+ 163us) client_negotiation.cc:769] Sending connection context
0619 06:45:06.287142 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 06:45:06.287480 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13969,"mutex_wait_us":1,"thread_start_us":14102,"threads_started":1}
I20260619 06:45:06.289034 24096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.230878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:52277 (local address 127.16.79.125:42515)
0619 06:45:06.248487 (+ 17609us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:06.248494 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:06.248512 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:45:06.248568 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:06.248573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:06.248703 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:06.250142 (+ 1439us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:06.250293 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.252130 (+ 1837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.272662 (+ 20532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.273372 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.277438 (+ 4066us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.280370 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:06.280378 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:06.280382 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:06.280429 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:06.284833 (+ 4404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.285677 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.285684 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.285690 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.285845 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.286377 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.286382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.286385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.286722 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:06.288327 (+ 1605us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:06.288453 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 06:45:06.288665 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17486,"thread_start_us":18107,"threads_started":1}
I20260619 06:45:06.290086 24089 heartbeater.cc:347] Connected to a master server at 127.16.79.125:42515
I20260619 06:45:06.290261 24089 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:06.290690 24089 heartbeater.cc:511] Master 127.16.79.125:42515 requested a full tablet report, sending...
I20260619 06:45:06.291996 23430 ts_manager.cc:194] Registered new tserver with Master: 1461b1bd05864c09bef1c0daae653d48 (127.16.79.67:35113)
I20260619 06:45:06.295882 24091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.253485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52690 (local address 127.16.79.126:41937)
0619 06:45:06.270009 (+ 16524us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:06.270014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:06.270032 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:45:06.270081 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:06.270086 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:06.270169 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:06.270262 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:06.278892 (+ 8630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.280016 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.282232 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.282880 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.283008 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.289173 (+ 6165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:06.289180 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:06.289185 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:06.289229 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:06.292600 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.293575 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.293583 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.293588 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.293749 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.294188 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.294194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.294198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.294584 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:06.294882 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:06.295257 (+ 375us) server_negotiation.cc:299] Negotiation successful
0619 06:45:06.295513 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16432,"thread_start_us":31285,"threads_started":1}
I20260619 06:45:06.300444 24103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.253750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50222 (local address 127.16.79.125:42515)
0619 06:45:06.279389 (+ 25639us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:06.279396 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:06.279414 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:45:06.279462 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:06.279467 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:06.279538 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:06.282093 (+ 2555us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:06.282783 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.284487 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.287624 (+ 3137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.288648 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.288771 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.293677 (+ 4906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:06.293682 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:06.293685 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:06.293718 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:06.296786 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.297701 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.297706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.297710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.297854 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.298322 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.298327 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.298332 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.298677 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:06.298899 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:06.299824 (+ 925us) server_negotiation.cc:299] Negotiation successful
0619 06:45:06.300042 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25506,"thread_start_us":25409,"threads_started":1}
I20260619 06:45:06.302896 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.264164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54368 (local address 127.16.79.124:37437)
0619 06:45:06.264385 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:06.264393 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:06.286014 (+ 21621us) server_negotiation.cc:407] Connection header received
0619 06:45:06.286085 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:06.286093 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:06.286203 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:06.286301 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:06.288874 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.289995 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.292126 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.292823 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.292889 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.296226 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:06.296232 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:06.296237 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:06.296275 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:06.299683 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.300778 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.300784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.300787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.300939 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.301736 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.301740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.301742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.301992 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:06.302169 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:06.302503 (+ 334us) server_negotiation.cc:299] Negotiation successful
0619 06:45:06.302658 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":14506,"threads_started":1}
I20260619 06:45:06.301173 22831 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52690:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:45:06.309131 22831 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:45:06.343575 24115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.328200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:35113 (local address 127.0.0.1:44840)
0619 06:45:06.329818 (+ 1618us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:06.329844 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:06.330783 (+ 939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:06.331999 (+ 1216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:06.332007 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:06.332043 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:06.332956 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:06.332974 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.335390 (+ 2416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.335395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:06.338810 (+ 3415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:06.338831 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.342424 (+ 3593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.342433 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:06.342705 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.342764 (+ 59us) client_negotiation.cc:769] Sending connection context
0619 06:45:06.342874 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 06:45:06.343100 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1427,"thread_start_us":1778,"threads_started":1}
I20260619 06:45:06.343595 24116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.328525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44840 (local address 127.16.79.67:35113)
0619 06:45:06.330620 (+ 2095us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:06.330628 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:06.330676 (+ 48us) server_negotiation.cc:407] Connection header received
0619 06:45:06.331004 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:06.331017 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:06.331354 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:06.331584 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:06.333474 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.335221 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.339055 (+ 3834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.342134 (+ 3079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.342383 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.342677 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:06.342956 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 06:45:06.343161 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1918,"thread_start_us":2196,"threads_started":1}
I20260619 06:45:06.350116 24022 tablet_service.cc:1467] Processing CreateTablet for tablet 19c00c9f46bb4ef5bf185075dd2c0206 (DEFAULT_TABLE table=test-table [id=ff55bbac33d2484da85e16098a3f24b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:06.351332 24115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.330386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:33659 (local address 127.0.0.1:52532)
0619 06:45:06.344048 (+ 13662us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:06.344068 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:06.344211 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:06.344972 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:06.344976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:06.345013 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:06.345216 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:06.345256 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.347403 (+ 2147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.347408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:06.349229 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:06.349248 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.350370 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.350375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:06.350552 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.350670 (+ 118us) client_negotiation.cc:769] Sending connection context
0619 06:45:06.350794 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 06:45:06.350963 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13551,"mutex_wait_us":110,"thread_start_us":14894,"threads_started":1}
I20260619 06:45:06.351727 24022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19c00c9f46bb4ef5bf185075dd2c0206. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:06.352061 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.330599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52532 (local address 127.16.79.66:33659)
0619 06:45:06.344486 (+ 13887us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:06.344492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:06.344515 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:45:06.344589 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:06.344594 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:06.344794 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:06.345013 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:06.345579 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.347220 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.349367 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.350218 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.350361 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.350559 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:06.351608 (+ 1049us) server_negotiation.cc:299] Negotiation successful
0619 06:45:06.351774 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12972,"thread_start_us":13608,"threads_started":1}
I20260619 06:45:06.354835 23880 tablet_service.cc:1467] Processing CreateTablet for tablet 19c00c9f46bb4ef5bf185075dd2c0206 (DEFAULT_TABLE table=test-table [id=ff55bbac33d2484da85e16098a3f24b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:06.355625 24117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.345851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:33147 (local address 127.0.0.1:42880)
0619 06:45:06.347258 (+ 1407us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:06.347277 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:06.347398 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:06.347963 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:06.347968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:06.347998 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:06.348418 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:06.348430 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.350185 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.350190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:06.352870 (+ 2680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:06.352886 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.354872 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.354880 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:06.355080 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.355113 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 06:45:06.355202 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 06:45:06.355329 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1272,"mutex_wait_us":1}
I20260619 06:45:06.356029 23880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19c00c9f46bb4ef5bf185075dd2c0206. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:06.356082 24119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.345932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42880 (local address 127.16.79.65:33147)
0619 06:45:06.346429 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:06.346436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:06.347372 (+ 936us) server_negotiation.cc:407] Connection header received
0619 06:45:06.347529 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:06.347536 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:06.347779 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:06.347968 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:06.348591 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.350005 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.353072 (+ 3067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.354430 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.354552 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.354749 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:06.355436 (+ 687us) server_negotiation.cc:299] Negotiation successful
0619 06:45:06.355593 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":1020,"threads_started":1}
I20260619 06:45:06.359388 23734 tablet_service.cc:1467] Processing CreateTablet for tablet 19c00c9f46bb4ef5bf185075dd2c0206 (DEFAULT_TABLE table=test-table [id=ff55bbac33d2484da85e16098a3f24b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:06.360706 23734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19c00c9f46bb4ef5bf185075dd2c0206. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:06.364539 24120 tablet_bootstrap.cc:492] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48: Bootstrap starting.
I20260619 06:45:06.368335 24120 tablet_bootstrap.cc:654] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:06.369437 24120 log.cc:824] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:06.372649 24120 tablet_bootstrap.cc:492] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48: No bootstrap required, opened a new log
I20260619 06:45:06.372980 24120 ts_tablet_manager.cc:1397] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260619 06:45:06.373657 24122 tablet_bootstrap.cc:492] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1: Bootstrap starting.
I20260619 06:45:06.376222 24123 tablet_bootstrap.cc:492] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417: Bootstrap starting.
I20260619 06:45:06.378706 24122 tablet_bootstrap.cc:654] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:06.379415 24123 tablet_bootstrap.cc:654] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:06.380230 24122 log.cc:824] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:06.380565 24123 log.cc:824] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:06.383180 24122 tablet_bootstrap.cc:492] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1: No bootstrap required, opened a new log
I20260619 06:45:06.383483 24122 ts_tablet_manager.cc:1397] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260619 06:45:06.384228 24120 raft_consensus.cc:348] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } }
I20260619 06:45:06.384644 24120 raft_consensus.cc:374] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:06.384747 24120 raft_consensus.cc:729] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1461b1bd05864c09bef1c0daae653d48, State: Initialized, Role: FOLLOWER
I20260619 06:45:06.385440 24120 consensus_queue.cc:260] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } }
I20260619 06:45:06.386209 24123 tablet_bootstrap.cc:492] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417: No bootstrap required, opened a new log
I20260619 06:45:06.386492 24123 ts_tablet_manager.cc:1397] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260619 06:45:06.387328 24120 ts_tablet_manager.cc:1428] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48: Time spent starting tablet: real 0.014s user 0.006s sys 0.008s
I20260619 06:45:06.387603 24088 heartbeater.cc:503] Master 127.16.79.126:41937 was elected leader, sending a full tablet report...
I20260619 06:45:06.396003 24122 raft_consensus.cc:348] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } }
I20260619 06:45:06.396550 24122 raft_consensus.cc:374] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:06.396649 24122 raft_consensus.cc:729] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db74f113a17d4d01959c109f47c628b1, State: Initialized, Role: FOLLOWER
I20260619 06:45:06.397339 24122 consensus_queue.cc:260] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } }
I20260619 06:45:06.398146 24123 raft_consensus.cc:348] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } }
I20260619 06:45:06.398550 24123 raft_consensus.cc:374] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:06.398635 24123 raft_consensus.cc:729] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48bb43759b34111970076634ed4b417, State: Initialized, Role: FOLLOWER
I20260619 06:45:06.399223 24122 ts_tablet_manager.cc:1428] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
I20260619 06:45:06.399230 24123 consensus_queue.cc:260] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } }
I20260619 06:45:06.399670 23946 heartbeater.cc:503] Master 127.16.79.126:41937 was elected leader, sending a full tablet report...
I20260619 06:45:06.404887 24127 raft_consensus.cc:482] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:45:06.405239 24127 raft_consensus.cc:504] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } }
I20260619 06:45:06.405568 24123 ts_tablet_manager.cc:1428] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417: Time spent starting tablet: real 0.019s user 0.014s sys 0.001s
I20260619 06:45:06.407217 24127 leader_election.cc:288] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1461b1bd05864c09bef1c0daae653d48 (127.16.79.67:35113), c48bb43759b34111970076634ed4b417 (127.16.79.65:33147)
I20260619 06:45:06.418195 24116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.407853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:56033 (local address 127.16.79.67:35113)
0619 06:45:06.408044 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:06.408050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:06.408830 (+ 780us) server_negotiation.cc:407] Connection header received
0619 06:45:06.409720 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:06.409729 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:06.409834 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:06.409911 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:06.411081 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.412265 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.415804 (+ 3539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.416982 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.417134 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.417221 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:06.417745 (+ 524us) server_negotiation.cc:299] Negotiation successful
0619 06:45:06.417884 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:45:06.419099 24129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.407726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:35113 (local address 127.16.79.66:56033)
0619 06:45:06.408419 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:06.408448 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:06.409314 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:06.410404 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:06.410412 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:06.410448 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:06.410922 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:06.410936 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.412433 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.412441 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:06.415554 (+ 3113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:06.415576 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.417364 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.417370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:06.417569 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.417602 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 06:45:06.418609 (+ 1007us) client_negotiation.cc:240] Negotiation successful
0619 06:45:06.418757 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":374,"threads_started":1}
I20260619 06:45:06.421074 24042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19c00c9f46bb4ef5bf185075dd2c0206" candidate_uuid: "db74f113a17d4d01959c109f47c628b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1461b1bd05864c09bef1c0daae653d48" is_pre_election: true
I20260619 06:45:06.421686 24042 raft_consensus.cc:2454] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db74f113a17d4d01959c109f47c628b1 in term 0.
I20260619 06:45:06.422420 23834 leader_election.cc:302] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1461b1bd05864c09bef1c0daae653d48, db74f113a17d4d01959c109f47c628b1; no voters:
I20260619 06:45:06.422873 24127 raft_consensus.cc:2781] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:45:06.423038 24127 raft_consensus.cc:482] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:45:06.423146 24127 raft_consensus.cc:3037] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:06.424840 24119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.408792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:39247 (local address 127.16.79.65:33147)
0619 06:45:06.408970 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:06.408976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:06.419516 (+ 10540us) server_negotiation.cc:407] Connection header received
0619 06:45:06.419687 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:06.419696 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:06.419789 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:06.419893 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:06.420276 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.421120 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.423154 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.423959 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.424197 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.424302 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:06.424569 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 06:45:06.424664 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 06:45:06.424983 24129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.408553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:33147 (local address 127.16.79.66:39247)
0619 06:45:06.419402 (+ 10849us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:06.419422 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:06.419548 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:06.419918 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:06.419922 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:06.419949 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:06.420103 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:06.420113 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.421265 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.421270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:06.422914 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:06.422928 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.424182 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.424191 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:06.424398 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.424440 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 06:45:06.424541 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 06:45:06.424708 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10747,"thread_start_us":11434,"threads_started":1}
I20260619 06:45:06.426043 24127 raft_consensus.cc:504] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } }
I20260619 06:45:06.426009 23754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19c00c9f46bb4ef5bf185075dd2c0206" candidate_uuid: "db74f113a17d4d01959c109f47c628b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48bb43759b34111970076634ed4b417" is_pre_election: true
I20260619 06:45:06.426553 23754 raft_consensus.cc:2454] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db74f113a17d4d01959c109f47c628b1 in term 0.
I20260619 06:45:06.426667 24127 leader_election.cc:288] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [CANDIDATE]: Term 1 election: Requested vote from peers 1461b1bd05864c09bef1c0daae653d48 (127.16.79.67:35113), c48bb43759b34111970076634ed4b417 (127.16.79.65:33147)
I20260619 06:45:06.427100 24042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19c00c9f46bb4ef5bf185075dd2c0206" candidate_uuid: "db74f113a17d4d01959c109f47c628b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1461b1bd05864c09bef1c0daae653d48"
I20260619 06:45:06.427168 23754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19c00c9f46bb4ef5bf185075dd2c0206" candidate_uuid: "db74f113a17d4d01959c109f47c628b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48bb43759b34111970076634ed4b417"
I20260619 06:45:06.427309 24042 raft_consensus.cc:3037] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:06.427429 23754 raft_consensus.cc:3037] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:06.430200 24042 raft_consensus.cc:2454] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db74f113a17d4d01959c109f47c628b1 in term 1.
I20260619 06:45:06.430590 23834 leader_election.cc:302] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1461b1bd05864c09bef1c0daae653d48, db74f113a17d4d01959c109f47c628b1; no voters:
I20260619 06:45:06.430723 23754 raft_consensus.cc:2454] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db74f113a17d4d01959c109f47c628b1 in term 1.
I20260619 06:45:06.430910 24127 raft_consensus.cc:2781] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:45:06.431433 24127 raft_consensus.cc:686] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [term 1 LEADER]: Becoming Leader. State: Replica: db74f113a17d4d01959c109f47c628b1, State: Running, Role: LEADER
I20260619 06:45:06.431953 24127 consensus_queue.cc:237] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } }
I20260619 06:45:06.437209 22831 catalog_manager.cc:5495] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to db74f113a17d4d01959c109f47c628b1 (127.16.79.66). New cstate: current_term: 1 leader_uuid: "db74f113a17d4d01959c109f47c628b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } health_report { overall_health: UNKNOWN } } }
I20260619 06:45:06.465191 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.460202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52546 (local address 127.16.79.66:33659)
0619 06:45:06.460408 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:06.460413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:06.460430 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:45:06.460557 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:06.460561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:06.460635 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:06.460752 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:45:06.461061 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.461738 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.462995 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.463386 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.463482 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.463915 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:45:06.464536 (+ 621us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:45:06.464696 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:06.464786 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 06:45:06.464869 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:45:06.473524 24042 raft_consensus.cc:1261] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48 [term 1 FOLLOWER]: Refusing update from remote peer db74f113a17d4d01959c109f47c628b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:45:06.473659 23754 raft_consensus.cc:1261] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417 [term 1 FOLLOWER]: Refusing update from remote peer db74f113a17d4d01959c109f47c628b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:45:06.474665 24127 consensus_queue.cc:1035] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:45:06.475059 24131 consensus_queue.cc:1035] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 06:45:06.478796 24090 tablet.cc:2369] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:45:06.510834 22830 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:45:06.511173 22830 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52690:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:45:06.517889 22830 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eeb9870492ab483c95bd19f6f547aa11: Sending DeleteTablet for 3 replicas of tablet 19c00c9f46bb4ef5bf185075dd2c0206
I20260619 06:45:06.519007 24022 tablet_service.cc:1514] Processing DeleteTablet for tablet 19c00c9f46bb4ef5bf185075dd2c0206 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:06 UTC) from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:44840
I20260619 06:45:06.519375 23880 tablet_service.cc:1514] Processing DeleteTablet for tablet 19c00c9f46bb4ef5bf185075dd2c0206 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:06 UTC) from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:52532
I20260619 06:45:06.520319 23734 tablet_service.cc:1514] Processing DeleteTablet for tablet 19c00c9f46bb4ef5bf185075dd2c0206 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:06 UTC) from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:42880
I20260619 06:45:06.527192 24144 tablet_replica.cc:330] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48: stopping tablet replica
I20260619 06:45:06.530203 24144 raft_consensus.cc:2229] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:45:06.530659 24145 tablet_replica.cc:330] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1: stopping tablet replica
I20260619 06:45:06.531144 24145 raft_consensus.cc:2229] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:45:06.531788 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 22799
I20260619 06:45:06.531816 24145 raft_consensus.cc:2258] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:06.531790 24144 raft_consensus.cc:2258] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:06.533106 22861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:06.534359 24144 ts_tablet_manager.cc:1905] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:06.537822 24146 tablet_replica.cc:330] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417: stopping tablet replica
I20260619 06:45:06.538410 24146 raft_consensus.cc:2229] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:45:06.538928 24146 raft_consensus.cc:2258] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:06.545576 24144 ts_tablet_manager.cc:1918] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:45:06.545775 24144 log.cc:1196] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/19c00c9f46bb4ef5bf185075dd2c0206
I20260619 06:45:06.546324 24144 ts_tablet_manager.cc:1939] T 19c00c9f46bb4ef5bf185075dd2c0206 P 1461b1bd05864c09bef1c0daae653d48: Deleting consensus metadata
I20260619 06:45:06.550051 24103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.522590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50232 (local address 127.16.79.125:42515)
0619 06:45:06.522841 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:06.522850 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:06.529848 (+ 6998us) server_negotiation.cc:407] Connection header received
0619 06:45:06.530318 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:06.530325 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:06.530412 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:06.530536 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:06.534766 (+ 4230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.535935 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.538323 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.538967 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.539098 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.542775 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:06.542783 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:06.542789 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:06.542834 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:06.546487 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.547633 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.547641 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.547645 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.547803 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.548298 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.548303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.548307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.548668 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:06.548897 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:06.549487 (+ 590us) server_negotiation.cc:299] Negotiation successful
0619 06:45:06.549718 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 06:45:06.550109 24146 ts_tablet_manager.cc:1905] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:06.553629 24145 ts_tablet_manager.cc:1905] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:06.557502 24146 ts_tablet_manager.cc:1918] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:45:06.557695 24146 log.cc:1196] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/19c00c9f46bb4ef5bf185075dd2c0206
I20260619 06:45:06.558307 24146 ts_tablet_manager.cc:1939] T 19c00c9f46bb4ef5bf185075dd2c0206 P c48bb43759b34111970076634ed4b417: Deleting consensus metadata
I20260619 06:45:06.560622 24145 ts_tablet_manager.cc:1918] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:45:06.560787 24145 log.cc:1196] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/19c00c9f46bb4ef5bf185075dd2c0206
I20260619 06:45:06.561323 24145 ts_tablet_manager.cc:1939] T 19c00c9f46bb4ef5bf185075dd2c0206 P db74f113a17d4d01959c109f47c628b1: Deleting consensus metadata
I20260619 06:45:06.568486 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.530038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54380 (local address 127.16.79.124:37437)
0619 06:45:06.530216 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:06.530224 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:06.551289 (+ 21065us) server_negotiation.cc:407] Connection header received
0619 06:45:06.551536 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:06.551541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:06.551622 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:06.551692 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:06.553551 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.554630 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.556566 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.557208 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.557305 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.561212 (+ 3907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:06.561221 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:06.561228 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:06.561291 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:06.564978 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.565952 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.565958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.565962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.566215 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.566677 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.566682 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.566686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.567151 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:06.567415 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:06.567803 (+ 388us) server_negotiation.cc:299] Negotiation successful
0619 06:45:06.568040 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 06:45:06.668969 22816 catalog_manager.cc:4841] TS db74f113a17d4d01959c109f47c628b1 (127.16.79.66:33659): tablet 19c00c9f46bb4ef5bf185075dd2c0206 (table test-table [id=ff55bbac33d2484da85e16098a3f24b4]) successfully deleted
I20260619 06:45:06.669353 22816 catalog_manager.cc:4841] TS c48bb43759b34111970076634ed4b417 (127.16.79.65:33147): tablet 19c00c9f46bb4ef5bf185075dd2c0206 (table test-table [id=ff55bbac33d2484da85e16098a3f24b4]) successfully deleted
I20260619 06:45:06.669426 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 22799
I20260619 06:45:06.711504 24094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.696132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54394 (local address 127.16.79.124:37437)
0619 06:45:06.696354 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:06.696362 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:06.696380 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:45:06.696441 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:06.696444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:06.696499 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:06.696617 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:06.698143 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.699000 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.700993 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.701433 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.701571 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.705272 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:06.705277 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:06.705280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:06.705311 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:06.708028 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.708862 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.708868 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.708874 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.709038 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.709465 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.709470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.709474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.709859 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:06.710131 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:06.710642 (+ 511us) server_negotiation.cc:299] Negotiation successful
0619 06:45:06.711111 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:45:06.712641 24103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:06.694599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50238 (local address 127.16.79.125:42515)
0619 06:45:06.694810 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:06.694818 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:06.695951 (+ 1133us) server_negotiation.cc:407] Connection header received
0619 06:45:06.696307 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:06.696314 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:06.696395 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:06.696465 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:06.698806 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.699608 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.701083 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:06.701773 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:06.701898 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:06.705242 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:06.705248 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:06.705254 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:06.705294 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:06.708787 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.709736 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.709741 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.709745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.709879 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:06.710533 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:06.710538 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:06.710542 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:06.710892 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:06.711126 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:06.711893 (+ 767us) server_negotiation.cc:299] Negotiation successful
0619 06:45:06.712258 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:45:07.498360 24155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:07.497344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.67:37461)
0619 06:45:07.497878 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:07.498044 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":318,"threads_started":1}
W20260619 06:45:07.498739 24088 heartbeater.cc:644] Failed to heartbeat to 127.16.79.126:41937 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
I20260619 06:45:07.504910 24156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:07.503836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.65:51291)
0619 06:45:07.504498 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:07.504659 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":496,"threads_started":1}
W20260619 06:45:07.505398 23800 heartbeater.cc:644] Failed to heartbeat to 127.16.79.126:41937 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
I20260619 06:45:07.505802 24157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:07.504647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.66:38655)
0619 06:45:07.505493 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:07.505618 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":643,"threads_started":1}
W20260619 06:45:07.506268 23946 heartbeater.cc:644] Failed to heartbeat to 127.16.79.126:41937 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
I20260619 06:45:08.018321 24158 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader eeb9870492ab483c95bd19f6f547aa11)
I20260619 06:45:08.018548 24158 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } }
I20260619 06:45:08.019110 24159 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader eeb9870492ab483c95bd19f6f547aa11)
I20260619 06:45:08.019457 24158 leader_election.cc:288] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eeb9870492ab483c95bd19f6f547aa11 (127.16.79.126:41937), 5e152d5cdd75450982491eddc2eddbcd (127.16.79.124:37437)
I20260619 06:45:08.019377 24159 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } }
I20260619 06:45:08.021938 24160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.019592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.0.0.1:53132)
0619 06:45:08.021449 (+ 1857us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.021660 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1710,"thread_start_us":1547,"threads_started":1}
W20260619 06:45:08.022213 23416 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
I20260619 06:45:08.024012 24163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.021991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.0.0.1:53146)
0619 06:45:08.022527 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.022759 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":243,"threads_started":1}
I20260619 06:45:08.024171 24160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.023780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.0.0.1:53162)
0619 06:45:08.023954 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.024021 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260619 06:45:08.024298 23605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
W20260619 06:45:08.024353 23416 leader_election.cc:334] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer eeb9870492ab483c95bd19f6f547aa11 (127.16.79.126:41937): Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
I20260619 06:45:08.028877 24159 leader_election.cc:288] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers eeb9870492ab483c95bd19f6f547aa11 (127.16.79.126:41937), a3a132b0e2d34fc7b4dcb59ab611b6d7 (127.16.79.125:42515)
I20260619 06:45:08.029786 24163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.029393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.0.0.1:53166)
0619 06:45:08.029536 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.029625 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 06:45:08.030095 23605 leader_election.cc:334] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer eeb9870492ab483c95bd19f6f547aa11 (127.16.79.126:41937): Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
I20260619 06:45:08.031257 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.019684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:37437 (local address 127.0.0.1:54400)
0619 06:45:08.021668 (+ 1984us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.021686 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:08.021897 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:08.023334 (+ 1437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:08.023344 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:08.023371 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:08.023724 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:08.023740 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.025802 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.025810 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:08.028663 (+ 2853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:08.028679 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.030436 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.030442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:08.030638 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.030664 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 06:45:08.030778 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 06:45:08.030910 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1867,"thread_start_us":1705,"threads_started":1}
I20260619 06:45:08.031322 24162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.020024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54400 (local address 127.16.79.124:37437)
0619 06:45:08.020950 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:08.020960 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:08.021774 (+ 814us) server_negotiation.cc:407] Connection header received
0619 06:45:08.022680 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:08.022689 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:08.022800 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:08.022882 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:08.024006 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.025430 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.028859 (+ 3429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.030275 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.030422 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.030512 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:08.030801 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 06:45:08.030928 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":738,"thread_start_us":593,"threads_started":1}
I20260619 06:45:08.031962 23629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5e152d5cdd75450982491eddc2eddbcd" is_pre_election: true
I20260619 06:45:08.032336 23629 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3a132b0e2d34fc7b4dcb59ab611b6d7 in term 1.
I20260619 06:45:08.032856 23414 leader_election.cc:302] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e152d5cdd75450982491eddc2eddbcd, a3a132b0e2d34fc7b4dcb59ab611b6d7; no voters: eeb9870492ab483c95bd19f6f547aa11
I20260619 06:45:08.033171 24158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 06:45:08.033298 24158 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader eeb9870492ab483c95bd19f6f547aa11)
I20260619 06:45:08.033442 24158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 1 FOLLOWER]: Advancing to term 2
I20260619 06:45:08.035820 24164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.022087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:42515 (local address 127.0.0.1:50246)
0619 06:45:08.022739 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.022760 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:08.022897 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:08.029720 (+ 6823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:08.029725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:08.029752 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:08.030281 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:08.030297 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.031831 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.031834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:08.033521 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:08.033538 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.035201 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.035233 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:08.035354 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.035379 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 06:45:08.035465 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 06:45:08.035556 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":3,"thread_start_us":1048,"threads_started":1}
I20260619 06:45:08.036018 24166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.023224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50246 (local address 127.16.79.125:42515)
0619 06:45:08.029356 (+ 6132us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:08.029361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:08.029384 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:45:08.029435 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:08.029439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:08.029520 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:08.029667 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:08.030497 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.031683 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.033663 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.035005 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.035141 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.035263 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:08.035514 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 06:45:08.035643 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6012,"thread_start_us":5824,"threads_started":1}
I20260619 06:45:08.036680 23439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e152d5cdd75450982491eddc2eddbcd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" is_pre_election: true
I20260619 06:45:08.036706 24158 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } }
I20260619 06:45:08.037194 23439 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e152d5cdd75450982491eddc2eddbcd in current term 2: Already voted for candidate a3a132b0e2d34fc7b4dcb59ab611b6d7 in this term.
I20260619 06:45:08.037611 24158 leader_election.cc:288] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [CANDIDATE]: Term 2 election: Requested vote from peers eeb9870492ab483c95bd19f6f547aa11 (127.16.79.126:41937), 5e152d5cdd75450982491eddc2eddbcd (127.16.79.124:37437)
I20260619 06:45:08.037875 23603 leader_election.cc:302] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e152d5cdd75450982491eddc2eddbcd; no voters: a3a132b0e2d34fc7b4dcb59ab611b6d7, eeb9870492ab483c95bd19f6f547aa11
I20260619 06:45:08.037876 23629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5e152d5cdd75450982491eddc2eddbcd"
I20260619 06:45:08.038147 23629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 1 FOLLOWER]: Advancing to term 2
I20260619 06:45:08.038269 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.037541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.0.0.1:53176)
0619 06:45:08.038056 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.038150 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355}
I20260619 06:45:08.039431 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.039063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.0.0.1:53180)
0619 06:45:08.039212 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.039293 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 06:45:08.039579 23416 leader_election.cc:334] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer eeb9870492ab483c95bd19f6f547aa11 (127.16.79.126:41937): Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
I20260619 06:45:08.040820 23629 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3a132b0e2d34fc7b4dcb59ab611b6d7 in term 2.
I20260619 06:45:08.040992 24159 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:45:08.041293 23414 leader_election.cc:302] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e152d5cdd75450982491eddc2eddbcd, a3a132b0e2d34fc7b4dcb59ab611b6d7; no voters: eeb9870492ab483c95bd19f6f547aa11
I20260619 06:45:08.041620 24158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 06:45:08.042168 24158 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [term 2 LEADER]: Becoming Leader. State: Replica: a3a132b0e2d34fc7b4dcb59ab611b6d7, State: Running, Role: LEADER
I20260619 06:45:08.042573 24158 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } }
I20260619 06:45:08.044842 24168 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3a132b0e2d34fc7b4dcb59ab611b6d7. Latest consensus state: current_term: 2 leader_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } } }
I20260619 06:45:08.045151 24168 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:08.045776 24170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:45:08.049172 24170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:45:08.049947 24170 catalog_manager.cc:1259] Loaded cluster ID: ba64cf8e50a9492cbc84bc1acc9f2cd9
I20260619 06:45:08.050040 24170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:45:08.051132 24170 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:45:08.051925 24170 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: Loaded TSK: 0
I20260619 06:45:08.052794 24170 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:45:08.098441 23430 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50238:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:45:08.105315 23430 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:45:08.109645 23629 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [term 2 FOLLOWER]: Refusing update from remote peer a3a132b0e2d34fc7b4dcb59ab611b6d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260619 06:45:08.109630 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.109099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.0.0.1:53206)
0619 06:45:08.109341 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.109434 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260619 06:45:08.110389 24168 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260619 06:45:08.110847 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.110366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.0.0.1:53214)
0619 06:45:08.110569 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.110691 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 06:45:08.111056 23416 consensus_peers.cc:487] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 -> Peer eeb9870492ab483c95bd19f6f547aa11 (127.16.79.126:41937): Couldn't send request to peer eeb9870492ab483c95bd19f6f547aa11. Status: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 06:45:08.117057 24159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3a132b0e2d34fc7b4dcb59ab611b6d7. Latest consensus state: current_term: 2 leader_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } } }
I20260619 06:45:08.117398 24159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:45:08.118472 24171 mvcc.cc:205] Tried to move back new op lower bound from 7298463777209790464 to 7298463776945729536. Current Snapshot: MvccSnapshot[applied={T|T < 7298463777209790464}]
I20260619 06:45:08.119521 24159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } } }
I20260619 06:45:08.119879 24159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e152d5cdd75450982491eddc2eddbcd [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:45:08.120471 24168 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "eeb9870492ab483c95bd19f6f547aa11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41937 } } peers { permanent_uuid: "a3a132b0e2d34fc7b4dcb59ab611b6d7" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 42515 } attrs { promote: false } } peers { permanent_uuid: "5e152d5cdd75450982491eddc2eddbcd" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 37437 } attrs { promote: false } } }
I20260619 06:45:08.120769 24168 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7 [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:08.142143 24179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.130079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:35113 (local address 127.0.0.1:44846)
0619 06:45:08.130745 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.130766 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:08.130889 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:08.136815 (+ 5926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:08.136819 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:08.136844 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:08.137177 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:08.137187 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.138552 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.138560 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:08.140062 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:08.140073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.141173 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.141179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:08.141401 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.141423 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 06:45:08.141520 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 06:45:08.141662 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":41,"thread_start_us":975,"threads_started":1}
I20260619 06:45:08.142762 24160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.129620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:33659 (local address 127.0.0.1:52560)
0619 06:45:08.129939 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.129955 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:08.130080 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:08.132164 (+ 2084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:08.132169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:08.132212 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:08.132564 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:08.132579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.135095 (+ 2516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.135104 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:08.138695 (+ 3591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:08.138713 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.140989 (+ 2276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.140998 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:08.141301 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.141406 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 06:45:08.141494 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 06:45:08.141708 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":6}
I20260619 06:45:08.143093 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.130339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44846 (local address 127.16.79.67:35113)
0619 06:45:08.136531 (+ 6192us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:08.136535 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:08.136551 (+ 16us) server_negotiation.cc:407] Connection header received
0619 06:45:08.136606 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:08.136610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:08.136676 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:08.136798 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:08.137467 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.138430 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.140236 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.140956 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.141105 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.141232 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:08.142599 (+ 1367us) server_negotiation.cc:299] Negotiation successful
0619 06:45:08.142789 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6088,"thread_start_us":5992,"threads_started":1}
I20260619 06:45:08.143762 24022 tablet_service.cc:1467] Processing CreateTablet for tablet 917b735de6bc43eb9e37e52ed29f020f (DEFAULT_TABLE table=test-table [id=85f961f626754095bd0b29e9aa2ba2d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:08.144002 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.129925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52560 (local address 127.16.79.66:33659)
0619 06:45:08.131707 (+ 1782us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:08.131717 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:08.131744 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:45:08.131839 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:08.131847 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:08.131960 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:08.132097 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:08.132762 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.134814 (+ 2052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.138978 (+ 4164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.140704 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.143166 (+ 2462us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.143316 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:08.143383 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 06:45:08.143598 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1616,"thread_start_us":1494,"threads_started":1}
I20260619 06:45:08.144251 24022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 917b735de6bc43eb9e37e52ed29f020f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:08.144780 23880 tablet_service.cc:1467] Processing CreateTablet for tablet 917b735de6bc43eb9e37e52ed29f020f (DEFAULT_TABLE table=test-table [id=85f961f626754095bd0b29e9aa2ba2d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:08.145210 23880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 917b735de6bc43eb9e37e52ed29f020f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:08.149451 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.129293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:33147 (local address 127.0.0.1:42888)
0619 06:45:08.129454 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.129477 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:08.129585 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:08.144864 (+ 15279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:08.144869 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:08.144893 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:08.145078 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:08.145090 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.146069 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.146076 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:08.147593 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:08.147608 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.148749 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.148755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:08.148937 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.148959 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 06:45:08.149066 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 06:45:08.149197 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 06:45:08.149662 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.129402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42888 (local address 127.16.79.65:33147)
0619 06:45:08.144567 (+ 15165us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:08.144573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:08.144600 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:45:08.144662 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:08.144666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:08.144723 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:08.144846 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:08.145241 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.145933 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.147774 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.148590 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.148734 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.148833 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:08.149093 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 06:45:08.149229 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15031,"thread_start_us":15545,"threads_started":1}
I20260619 06:45:08.150276 23734 tablet_service.cc:1467] Processing CreateTablet for tablet 917b735de6bc43eb9e37e52ed29f020f (DEFAULT_TABLE table=test-table [id=85f961f626754095bd0b29e9aa2ba2d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:08.150591 23734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 917b735de6bc43eb9e37e52ed29f020f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:08.151217 24181 tablet_bootstrap.cc:492] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48: Bootstrap starting.
I20260619 06:45:08.152966 24181 tablet_bootstrap.cc:654] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:08.155591 24182 tablet_bootstrap.cc:492] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1: Bootstrap starting.
I20260619 06:45:08.157109 24182 tablet_bootstrap.cc:654] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:08.159045 24182 tablet_bootstrap.cc:492] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1: No bootstrap required, opened a new log
I20260619 06:45:08.159089 24185 tablet_bootstrap.cc:492] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417: Bootstrap starting.
I20260619 06:45:08.159205 24182 ts_tablet_manager.cc:1397] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260619 06:45:08.160012 24182 raft_consensus.cc:348] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } }
I20260619 06:45:08.160274 24181 tablet_bootstrap.cc:492] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48: No bootstrap required, opened a new log
I20260619 06:45:08.160292 24182 raft_consensus.cc:374] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:08.160377 24182 raft_consensus.cc:729] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db74f113a17d4d01959c109f47c628b1, State: Initialized, Role: FOLLOWER
I20260619 06:45:08.160413 24181 ts_tablet_manager.cc:1397] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260619 06:45:08.160615 24182 consensus_queue.cc:260] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } }
I20260619 06:45:08.160948 24185 tablet_bootstrap.cc:654] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:08.161404 24181 raft_consensus.cc:348] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } }
I20260619 06:45:08.161608 24181 raft_consensus.cc:374] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:08.161716 24181 raft_consensus.cc:729] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1461b1bd05864c09bef1c0daae653d48, State: Initialized, Role: FOLLOWER
I20260619 06:45:08.161953 24181 consensus_queue.cc:260] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } }
I20260619 06:45:08.162806 24181 ts_tablet_manager.cc:1428] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 06:45:08.163460 24185 tablet_bootstrap.cc:492] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417: No bootstrap required, opened a new log
I20260619 06:45:08.163653 24185 ts_tablet_manager.cc:1397] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 06:45:08.164479 24189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.163391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.67:38215)
0619 06:45:08.164033 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.164223 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":331,"threads_started":1}
I20260619 06:45:08.165079 24182 ts_tablet_manager.cc:1428] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260619 06:45:08.165098 24185 raft_consensus.cc:348] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } }
I20260619 06:45:08.165318 24185 raft_consensus.cc:374] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:08.165468 24185 raft_consensus.cc:729] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48bb43759b34111970076634ed4b417, State: Initialized, Role: FOLLOWER
I20260619 06:45:08.165778 24185 consensus_queue.cc:260] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } }
I20260619 06:45:08.166706 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.165615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.66:57401)
0619 06:45:08.166259 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.166422 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":250,"threads_started":1}
I20260619 06:45:08.168540 24089 heartbeater.cc:503] Master 127.16.79.125:42515 was elected leader, sending a full tablet report...
I20260619 06:45:08.169157 24189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.167881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.67:49441)
0619 06:45:08.168939 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.169021 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":951}
I20260619 06:45:08.170209 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.169825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.66:43295)
0619 06:45:08.169978 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.170063 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 06:45:08.171139 23947 heartbeater.cc:503] Master 127.16.79.125:42515 was elected leader, sending a full tablet report...
I20260619 06:45:08.171939 24185 ts_tablet_manager.cc:1428] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260619 06:45:08.173120 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.172103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.65:57413)
0619 06:45:08.172706 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.172876 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":275,"threads_started":1}
I20260619 06:45:08.174252 23801 heartbeater.cc:503] Master 127.16.79.125:42515 was elected leader, sending a full tablet report...
I20260619 06:45:08.174480 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.173715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.65:49039)
0619 06:45:08.174247 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.174334 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"threads_started":1}
W20260619 06:45:08.280857 23802 tablet.cc:2369] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:45:08.383502 23948 tablet.cc:2369] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:45:08.395232 24188 raft_consensus.cc:482] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:45:08.395525 24188 raft_consensus.cc:504] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } }
I20260619 06:45:08.397141 24188 leader_election.cc:288] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c48bb43759b34111970076634ed4b417 (127.16.79.65:33147), db74f113a17d4d01959c109f47c628b1 (127.16.79.66:33659)
I20260619 06:45:08.405428 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.397964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:51573 (local address 127.16.79.66:33659)
0619 06:45:08.398170 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:08.398178 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:08.398580 (+ 402us) server_negotiation.cc:407] Connection header received
0619 06:45:08.398716 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:08.398719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:08.398776 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:08.398847 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:08.399250 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.400252 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.403052 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.404341 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.404514 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.404588 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:08.405029 (+ 441us) server_negotiation.cc:299] Negotiation successful
0619 06:45:08.405165 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:45:08.405426 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.397537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:47343 (local address 127.16.79.65:33147)
0619 06:45:08.397752 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:08.397760 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:08.397782 (+ 22us) server_negotiation.cc:407] Connection header received
0619 06:45:08.398017 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:08.398026 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:08.398128 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:08.398240 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:08.398962 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.400022 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.403052 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.404341 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.404514 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.404588 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:08.405029 (+ 441us) server_negotiation.cc:299] Negotiation successful
0619 06:45:08.405162 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:45:08.405798 24189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.397421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:33147 (local address 127.16.79.67:47343)
0619 06:45:08.397612 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.397635 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:08.397834 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:08.398276 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:08.398281 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:08.398311 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:08.398589 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:08.398602 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.400331 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.400335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:08.402855 (+ 2520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:08.402870 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.404560 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.404567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:08.404824 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.404877 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 06:45:08.404976 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 06:45:08.405188 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":271}
I20260619 06:45:08.405835 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.397843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:33659 (local address 127.16.79.67:51573)
0619 06:45:08.398465 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.398488 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:08.398611 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:08.398875 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:08.398878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:08.398895 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:08.399090 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:08.399096 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.400411 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.400415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:08.402855 (+ 2440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:08.402870 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.404560 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.404567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:08.404819 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.404885 (+ 66us) client_negotiation.cc:769] Sending connection context
0619 06:45:08.404976 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 06:45:08.405190 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":252,"thread_start_us":347,"threads_started":1}
I20260619 06:45:08.406463 23754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917b735de6bc43eb9e37e52ed29f020f" candidate_uuid: "1461b1bd05864c09bef1c0daae653d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48bb43759b34111970076634ed4b417" is_pre_election: true
I20260619 06:45:08.406694 23754 raft_consensus.cc:2454] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1461b1bd05864c09bef1c0daae653d48 in term 0.
I20260619 06:45:08.406733 23900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917b735de6bc43eb9e37e52ed29f020f" candidate_uuid: "1461b1bd05864c09bef1c0daae653d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db74f113a17d4d01959c109f47c628b1" is_pre_election: true
I20260619 06:45:08.407187 23976 leader_election.cc:302] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1461b1bd05864c09bef1c0daae653d48, c48bb43759b34111970076634ed4b417; no voters:
I20260619 06:45:08.407217 23900 raft_consensus.cc:2454] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1461b1bd05864c09bef1c0daae653d48 in term 0.
I20260619 06:45:08.407565 24188 raft_consensus.cc:2781] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:45:08.407697 24188 raft_consensus.cc:482] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:45:08.407831 24188 raft_consensus.cc:3037] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:08.410112 24188 raft_consensus.cc:504] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } }
I20260619 06:45:08.410710 24188 leader_election.cc:288] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [CANDIDATE]: Term 1 election: Requested vote from peers c48bb43759b34111970076634ed4b417 (127.16.79.65:33147), db74f113a17d4d01959c109f47c628b1 (127.16.79.66:33659)
I20260619 06:45:08.411156 23754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917b735de6bc43eb9e37e52ed29f020f" candidate_uuid: "1461b1bd05864c09bef1c0daae653d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c48bb43759b34111970076634ed4b417"
I20260619 06:45:08.411204 23900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "917b735de6bc43eb9e37e52ed29f020f" candidate_uuid: "1461b1bd05864c09bef1c0daae653d48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db74f113a17d4d01959c109f47c628b1"
I20260619 06:45:08.411357 23754 raft_consensus.cc:3037] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:08.411406 23900 raft_consensus.cc:3037] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:08.413597 23754 raft_consensus.cc:2454] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1461b1bd05864c09bef1c0daae653d48 in term 1.
I20260619 06:45:08.413795 23900 raft_consensus.cc:2454] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1461b1bd05864c09bef1c0daae653d48 in term 1.
I20260619 06:45:08.413959 23976 leader_election.cc:302] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1461b1bd05864c09bef1c0daae653d48, c48bb43759b34111970076634ed4b417; no voters:
I20260619 06:45:08.414234 24188 raft_consensus.cc:2781] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:45:08.414645 24188 raft_consensus.cc:686] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [term 1 LEADER]: Becoming Leader. State: Replica: 1461b1bd05864c09bef1c0daae653d48, State: Running, Role: LEADER
I20260619 06:45:08.415059 24188 consensus_queue.cc:237] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } } peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } }
I20260619 06:45:08.415599 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.415181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.67:45925)
0619 06:45:08.415333 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.415475 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 06:45:08.418569 23429 catalog_manager.cc:5495] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1461b1bd05864c09bef1c0daae653d48 (127.16.79.67). New cstate: current_term: 1 leader_uuid: "1461b1bd05864c09bef1c0daae653d48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1461b1bd05864c09bef1c0daae653d48" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 35113 } health_report { overall_health: HEALTHY } } }
I20260619 06:45:08.435587 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.430455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44852 (local address 127.16.79.67:35113)
0619 06:45:08.430626 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:08.430633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:08.430672 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:45:08.430916 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:08.430920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:08.430992 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:08.431139 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:45:08.431491 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.432260 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.433511 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.433920 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.434046 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.434476 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:45:08.434989 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:45:08.435142 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:08.435237 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 06:45:08.435315 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 06:45:08.439623 23754 raft_consensus.cc:1261] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417 [term 1 FOLLOWER]: Refusing update from remote peer 1461b1bd05864c09bef1c0daae653d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:45:08.439986 23900 raft_consensus.cc:1261] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1 [term 1 FOLLOWER]: Refusing update from remote peer 1461b1bd05864c09bef1c0daae653d48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:45:08.440367 24195 consensus_queue.cc:1035] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c48bb43759b34111970076634ed4b417" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:45:08.440747 24188 consensus_queue.cc:1035] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db74f113a17d4d01959c109f47c628b1" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:45:08.443490 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.442790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.66:52221)
0619 06:45:08.443111 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.443263 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260619 06:45:08.444775 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.442901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.65:37467)
0619 06:45:08.443878 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.444461 (+ 583us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":760}
I20260619 06:45:08.449832 24199 mvcc.cc:205] Tried to move back new op lower bound from 7298463778561474560 to 7298463778472095744. Current Snapshot: MvccSnapshot[applied={T|T < 7298463778561474560}]
I20260619 06:45:08.450546 24200 mvcc.cc:205] Tried to move back new op lower bound from 7298463778561474560 to 7298463778472095744. Current Snapshot: MvccSnapshot[applied={T|T < 7298463778561474560}]
I20260619 06:45:08.451001 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.450143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.66:59805)
0619 06:45:08.450660 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.450778 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355}
I20260619 06:45:08.452528 24193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.451677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.65:53755)
0619 06:45:08.452214 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.452328 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370}
I20260619 06:45:08.457464 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.456983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.67:54431)
0619 06:45:08.457179 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.457288 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 06:45:08.459614 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.459054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.16.79.67:53743)
0619 06:45:08.459352 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.459470 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189}
I20260619 06:45:08.467039 23430 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:45:08.467270 23430 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:45:08.472710 23430 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a3a132b0e2d34fc7b4dcb59ab611b6d7: Sending DeleteTablet for 3 replicas of tablet 917b735de6bc43eb9e37e52ed29f020f
I20260619 06:45:08.473845 23734 tablet_service.cc:1514] Processing DeleteTablet for tablet 917b735de6bc43eb9e37e52ed29f020f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:08 UTC) from {username='oryx', principal='oryx/127.16.79.125@KRBTEST.COM'} at 127.0.0.1:42888
I20260619 06:45:08.474006 23880 tablet_service.cc:1514] Processing DeleteTablet for tablet 917b735de6bc43eb9e37e52ed29f020f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:08 UTC) from {username='oryx', principal='oryx/127.16.79.125@KRBTEST.COM'} at 127.0.0.1:52560
I20260619 06:45:08.474640 24022 tablet_service.cc:1514] Processing DeleteTablet for tablet 917b735de6bc43eb9e37e52ed29f020f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:08 UTC) from {username='oryx', principal='oryx/127.16.79.125@KRBTEST.COM'} at 127.0.0.1:44846
I20260619 06:45:08.475265 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 23607
I20260619 06:45:08.485217 24210 tablet_replica.cc:330] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1: stopping tablet replica
I20260619 06:45:08.485491 24210 raft_consensus.cc:2229] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:45:08.485885 24210 raft_consensus.cc:2258] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:08.488520 24210 ts_tablet_manager.cc:1905] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:08.489948 24211 tablet_replica.cc:330] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48: stopping tablet replica
I20260619 06:45:08.490146 24211 raft_consensus.cc:2229] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:45:08.490792 24211 raft_consensus.cc:2258] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:08.491987 24211 ts_tablet_manager.cc:1905] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:08.493675 24209 tablet_replica.cc:330] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417: stopping tablet replica
I20260619 06:45:08.493865 24209 raft_consensus.cc:2229] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:45:08.494156 24209 raft_consensus.cc:2258] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:08.495452 24209 ts_tablet_manager.cc:1905] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:08.497541 24210 ts_tablet_manager.cc:1918] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:45:08.497684 24210 log.cc:1196] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/917b735de6bc43eb9e37e52ed29f020f
I20260619 06:45:08.498159 24210 ts_tablet_manager.cc:1939] T 917b735de6bc43eb9e37e52ed29f020f P db74f113a17d4d01959c109f47c628b1: Deleting consensus metadata
I20260619 06:45:08.499338 23415 catalog_manager.cc:4841] TS db74f113a17d4d01959c109f47c628b1 (127.16.79.66:33659): tablet 917b735de6bc43eb9e37e52ed29f020f (table test-table [id=85f961f626754095bd0b29e9aa2ba2d6]) successfully deleted
I20260619 06:45:08.500265 24209 ts_tablet_manager.cc:1918] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:45:08.500394 24209 log.cc:1196] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/917b735de6bc43eb9e37e52ed29f020f
I20260619 06:45:08.500708 24211 ts_tablet_manager.cc:1918] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:45:08.500777 24209 ts_tablet_manager.cc:1939] T 917b735de6bc43eb9e37e52ed29f020f P c48bb43759b34111970076634ed4b417: Deleting consensus metadata
I20260619 06:45:08.500855 24211 log.cc:1196] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/917b735de6bc43eb9e37e52ed29f020f
I20260619 06:45:08.501394 24211 ts_tablet_manager.cc:1939] T 917b735de6bc43eb9e37e52ed29f020f P 1461b1bd05864c09bef1c0daae653d48: Deleting consensus metadata
I20260619 06:45:08.501677 23415 catalog_manager.cc:4841] TS c48bb43759b34111970076634ed4b417 (127.16.79.65:33147): tablet 917b735de6bc43eb9e37e52ed29f020f (table test-table [id=85f961f626754095bd0b29e9aa2ba2d6]) successfully deleted
I20260619 06:45:08.502499 23415 catalog_manager.cc:4841] TS 1461b1bd05864c09bef1c0daae653d48 (127.16.79.67:35113): tablet 917b735de6bc43eb9e37e52ed29f020f (table test-table [id=85f961f626754095bd0b29e9aa2ba2d6]) successfully deleted
I20260619 06:45:08.510365 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.478356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42904 (local address 127.16.79.65:33147)
0619 06:45:08.478586 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:08.478595 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:08.486918 (+ 8323us) server_negotiation.cc:407] Connection header received
0619 06:45:08.487468 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:08.487473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:08.487556 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:08.487633 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:08.491107 (+ 3474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.492244 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.494161 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.494885 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.495008 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.504337 (+ 9329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:08.504350 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:08.504366 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:08.504399 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:08.507167 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:08.507875 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:08.507882 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:08.507884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:08.507993 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:08.508407 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:08.508414 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:08.508422 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:08.508817 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:08.509180 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:08.509699 (+ 519us) server_negotiation.cc:299] Negotiation successful
0619 06:45:08.509977 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 06:45:08.511134 23794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:08.516894 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.516378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.0.0.1:53218)
0619 06:45:08.516570 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.516697 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 06:45:08.518162 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.517700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41937 (local address 127.0.0.1:53234)
0619 06:45:08.517908 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:08.518002 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 06:45:08.614023 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 23607
I20260619 06:45:08.631825 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 23811
I20260619 06:45:08.650357 24178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.632836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52570 (local address 127.16.79.66:33659)
0619 06:45:08.632998 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:08.633005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:08.633070 (+ 65us) server_negotiation.cc:407] Connection header received
0619 06:45:08.633296 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:08.633301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:08.633375 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:08.633485 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:08.634868 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.635615 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.636950 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.637374 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.637504 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.644208 (+ 6704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:08.644226 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:08.644242 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:08.644292 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:08.647394 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:08.648174 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:08.648184 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:08.648189 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:08.648325 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:08.648703 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:08.648708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:08.648715 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:08.649100 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:08.649408 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:08.649724 (+ 316us) server_negotiation.cc:299] Negotiation successful
0619 06:45:08.649946 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 06:45:08.651068 23940 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:08.769352 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 23811
I20260619 06:45:08.787155 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 23958
I20260619 06:45:08.804736 24180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:08.788086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44866 (local address 127.16.79.67:35113)
0619 06:45:08.788308 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:08.788313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:08.788334 (+ 21us) server_negotiation.cc:407] Connection header received
0619 06:45:08.788549 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:08.788553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:08.788645 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:08.788725 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:08.790821 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.791641 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.793974 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:08.794439 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:08.794549 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:08.798082 (+ 3533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:08.798096 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:08.798111 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:08.798148 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:08.801346 (+ 3198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:08.802232 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:08.802240 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:08.802242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:08.802411 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:08.802933 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:08.802937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:08.802938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:08.803298 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:08.803652 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:08.804059 (+ 407us) server_negotiation.cc:299] Negotiation successful
0619 06:45:08.804344 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260619 06:45:08.805562 24082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:08.933534 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 23958
I20260619 06:45:08.954238 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 23292
I20260619 06:45:08.955310 23460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:09.076277 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 23292
I20260619 06:45:09.097003 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 23394
I20260619 06:45:09.098111 23650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:09.211980 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 23394
2026-06-19T06:45:09Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9537 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:45:09 dist-test-slave-5w2x krb5kdc[24220](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:45:09 dist-test-slave-5w2x krb5kdc[24220](info): set up 2 sockets
Jun 19 06:45:09 dist-test-slave-5w2x krb5kdc[24220](info): commencing operation
krb5kdc: starting...
W20260619 06:45:10.333181 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:45:10 dist-test-slave-5w2x krb5kdc[24220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:45:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:45:10Z Disabled control of system clock
WARNING: no policy specified for oryx/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.126.keytab.
Entry for principal oryx/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.126.keytab.
I20260619 06:45:10.548067 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:38019
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.126.keytab
--principal=oryx/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:43455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:38019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:10.871449 24233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:10.871737 24233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:10.871888 24233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:10.879161 24233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:45:10.879271 24233 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 06:45:10.879359 24233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:10.879468 24233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:10.879545 24233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:45:10.879635 24233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:45:10.889631 24233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:43455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:38019
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.126.keytab
--principal=oryx/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:38019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:10.890913 24233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:10.892280 24233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:10.904111 24238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:10.904251 24239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:10.904750 24241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:10.905211 24233 server_base.cc:1034] running on GCE node
I20260619 06:45:10.906039 24233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:10.907367 24233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:10.908615 24233 hybrid_clock.cc:648] HybridClock initialized: now 1781851510908560 us; error 27 us; skew 500 ppm
I20260619 06:45:10.908946 24233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:10 dist-test-slave-5w2x krb5kdc[24220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851510, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:10.911986 24233 init.cc:377] Logged in from keytab as oryx/127.16.79.126@KRBTEST.COM (short username oryx)
I20260619 06:45:10.913470 24233 webserver.cc:460] Webserver started at http://127.16.79.126:36457/ using document root <none> and password file <none>
I20260619 06:45:10.914178 24233 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:10.914302 24233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:10.914616 24233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:10.917622 24233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "1faa3e74ca1f4504a9b45a6bc193c84b"
format_stamp: "Formatted at 2026-06-19 06:45:10 on dist-test-slave-5w2x"
server_key: "232248a7c1b5f92b8cf4d7411f8bf5e5"
server_key_iv: "758a851ecd8beb9e38a8147941de3cd7"
server_key_version: "encryptionkey@0"
I20260619 06:45:10.918409 24233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "1faa3e74ca1f4504a9b45a6bc193c84b"
format_stamp: "Formatted at 2026-06-19 06:45:10 on dist-test-slave-5w2x"
server_key: "232248a7c1b5f92b8cf4d7411f8bf5e5"
server_key_iv: "758a851ecd8beb9e38a8147941de3cd7"
server_key_version: "encryptionkey@0"
I20260619 06:45:10.923477 24233 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:45:10.926959 24248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:10.928436 24233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:45:10.928610 24233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "1faa3e74ca1f4504a9b45a6bc193c84b"
format_stamp: "Formatted at 2026-06-19 06:45:10 on dist-test-slave-5w2x"
server_key: "232248a7c1b5f92b8cf4d7411f8bf5e5"
server_key_iv: "758a851ecd8beb9e38a8147941de3cd7"
server_key_version: "encryptionkey@0"
I20260619 06:45:10.928828 24233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:10.942657 24233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:10.947234 24233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:10.947525 24233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:10.967797 24233 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:38019
I20260619 06:45:10.967821 24309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:38019 every 8 connection(s)
I20260619 06:45:10.969225 24233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:45:10.972958 24310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:10.978546 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 24233
I20260619 06:45:10.978749 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:45:10.979374 16701 external_mini_cluster.cc:1428] Setting key 0908628deb9fd301a6defd6b35a1dfcf
I20260619 06:45:10.983274 24310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b: Bootstrap starting.
I20260619 06:45:10.987107 24310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:10.988253 24310 log.cc:824] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:10.991392 24310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b: No bootstrap required, opened a new log
Jun 19 06:45:10 dist-test-slave-5w2x krb5kdc[24220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM
I20260619 06:45:10.999637 24310 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faa3e74ca1f4504a9b45a6bc193c84b" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38019 } }
I20260619 06:45:11.000034 24310 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:11.000159 24310 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1faa3e74ca1f4504a9b45a6bc193c84b, State: Initialized, Role: FOLLOWER
I20260619 06:45:11.000818 24310 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faa3e74ca1f4504a9b45a6bc193c84b" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38019 } }
I20260619 06:45:11.001072 24310 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:45:11.001205 24310 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:45:11.001397 24310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:11.002609 24313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:10.981310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46132 (local address 127.16.79.126:38019)
0619 06:45:10.982113 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:10.982125 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:10.982166 (+ 41us) server_negotiation.cc:407] Connection header received
0619 06:45:10.983864 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:10.983888 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:10.984343 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:10.984682 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:10.986544 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:10.988048 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:10.989724 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:10.990398 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:10.990533 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:10.995094 (+ 4561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:10.995101 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:10.995120 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:10.995159 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:10.998330 (+ 3171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:10.999026 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:10.999033 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:10.999040 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:10.999195 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:10.999567 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:10.999573 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:10.999576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:11.000179 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:11.000461 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:11.000911 (+ 450us) server_negotiation.cc:299] Negotiation successful
0619 06:45:11.001537 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":382,"threads_started":1}
I20260619 06:45:11.003856 24310 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faa3e74ca1f4504a9b45a6bc193c84b" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38019 } }
I20260619 06:45:11.004438 24310 leader_election.cc:302] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1faa3e74ca1f4504a9b45a6bc193c84b; no voters:
I20260619 06:45:11.005090 24310 leader_election.cc:288] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:45:11.005193 24315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:45:11.006261 24315 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [term 1 LEADER]: Becoming Leader. State: Replica: 1faa3e74ca1f4504a9b45a6bc193c84b, State: Running, Role: LEADER
I20260619 06:45:11.006731 24315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faa3e74ca1f4504a9b45a6bc193c84b" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38019 } }
I20260619 06:45:11.007130 24310 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:45:11.009414 24317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1faa3e74ca1f4504a9b45a6bc193c84b. Latest consensus state: current_term: 1 leader_uuid: "1faa3e74ca1f4504a9b45a6bc193c84b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faa3e74ca1f4504a9b45a6bc193c84b" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38019 } } }
I20260619 06:45:11.009761 24317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:11.010568 24316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1faa3e74ca1f4504a9b45a6bc193c84b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1faa3e74ca1f4504a9b45a6bc193c84b" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38019 } } }
I20260619 06:45:11.010901 24316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:11.023933 24321 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:45:11.030030 24321 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:45:11.039661 24321 catalog_manager.cc:1347] Generated new cluster ID: 85e4b32c04a643b4b6c1d6e84ac8b840
I20260619 06:45:11.039783 24321 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:45:11.089046 24321 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:45:11.090389 24321 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:45:11.097995 24321 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b: Generated new TSK 0
I20260619 06:45:11.099027 24321 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.65.keytab.
Entry for principal oryx/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.65.keytab.
I20260619 06:45:11.157464 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:38019
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.65.keytab
--principal=oryx/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:43455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:11.489776 24337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:11.490042 24337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:11.490223 24337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:11.497269 24337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:11.497394 24337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:11.497540 24337 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:45:11.497670 24337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:45:11.507712 24337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:43455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.65.keytab
--principal=oryx/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.79.126:38019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:11.509020 24337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:11.510309 24337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:11.522941 24342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:11.522872 24343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:11.522872 24345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:11.524039 24337 server_base.cc:1034] running on GCE node
I20260619 06:45:11.524703 24337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:11.525658 24337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:11.526880 24337 hybrid_clock.cc:648] HybridClock initialized: now 1781851511526818 us; error 36 us; skew 500 ppm
I20260619 06:45:11.527210 24337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:11 dist-test-slave-5w2x krb5kdc[24220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851511, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:11.530383 24337 init.cc:377] Logged in from keytab as oryx/127.16.79.65@KRBTEST.COM (short username oryx)
I20260619 06:45:11.531772 24337 webserver.cc:460] Webserver started at http://127.16.79.65:46839/ using document root <none> and password file <none>
I20260619 06:45:11.532548 24337 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:11.532641 24337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:11.532914 24337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:11.535836 24337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "9552fb76d538450791575265a9240c9e"
format_stamp: "Formatted at 2026-06-19 06:45:11 on dist-test-slave-5w2x"
server_key: "47864b54b8e25fa1dfeea1af63778647"
server_key_iv: "b0d9121856f47c198c87b7b3c0218b3c"
server_key_version: "encryptionkey@0"
I20260619 06:45:11.536655 24337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "9552fb76d538450791575265a9240c9e"
format_stamp: "Formatted at 2026-06-19 06:45:11 on dist-test-slave-5w2x"
server_key: "47864b54b8e25fa1dfeea1af63778647"
server_key_iv: "b0d9121856f47c198c87b7b3c0218b3c"
server_key_version: "encryptionkey@0"
I20260619 06:45:11.541565 24337 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260619 06:45:11.544888 24352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:11.546443 24337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:45:11.546623 24337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "9552fb76d538450791575265a9240c9e"
format_stamp: "Formatted at 2026-06-19 06:45:11 on dist-test-slave-5w2x"
server_key: "47864b54b8e25fa1dfeea1af63778647"
server_key_iv: "b0d9121856f47c198c87b7b3c0218b3c"
server_key_version: "encryptionkey@0"
I20260619 06:45:11.546850 24337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:11.565344 24337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:11.569557 24337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:11.569833 24337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:11.572531 24337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:11.572651 24337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:11.572808 24337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:11.572865 24337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 06:45:11 dist-test-slave-5w2x krb5kdc[24220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851511, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM
I20260619 06:45:11.595446 24360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:11.572869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:55811 (local address 127.16.79.126:38019)
0619 06:45:11.573301 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:11.573309 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:11.574529 (+ 1220us) server_negotiation.cc:407] Connection header received
0619 06:45:11.576481 (+ 1952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:11.576486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:11.576574 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:11.576699 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:11.578960 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:11.580274 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:11.582680 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:11.583457 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:11.583603 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:11.588802 (+ 5199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:11.588811 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:11.588816 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:11.588862 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:11.591997 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:11.592996 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:11.593002 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:11.593005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:11.593186 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:11.593753 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:11.593758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:11.593762 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:11.594035 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:11.594189 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:11.594867 (+ 678us) server_negotiation.cc:299] Negotiation successful
0619 06:45:11.595078 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":223,"threads_started":1}
I20260619 06:45:11.596611 24362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:11.573605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38019 (local address 127.16.79.65:55811)
0619 06:45:11.574385 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:11.574422 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:11.576220 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:11.576939 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:11.576955 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:11.577856 (+ 901us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:11.578747 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:11.578775 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:11.580445 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:11.580453 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:11.582518 (+ 2065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:11.582539 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:11.583729 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:11.583738 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:11.584032 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:11.584871 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:11.584910 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:11.588495 (+ 3585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:11.592163 (+ 3668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:11.592172 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:11.592198 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:11.592846 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:11.593344 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:11.593352 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:11.593356 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:11.593591 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:11.594168 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:11.594176 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:11.594515 (+ 339us) client_negotiation.cc:769] Sending connection context
0619 06:45:11.594773 (+ 258us) client_negotiation.cc:240] Negotiation successful
0619 06:45:11.595295 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":324,"threads_started":1}
I20260619 06:45:11.621074 24360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:11.606230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:55779 (local address 127.16.79.126:38019)
0619 06:45:11.606456 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:11.606466 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:11.607663 (+ 1197us) server_negotiation.cc:407] Connection header received
0619 06:45:11.607868 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:11.607903 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:11.608016 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:11.608210 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:11.609522 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:11.610561 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:11.612512 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:11.612914 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:11.613049 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:11.615483 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:11.615489 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:11.615494 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:11.615531 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:11.618515 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:11.619173 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:11.619179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:11.619183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:11.619308 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:11.619662 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:11.619667 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:11.619670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:11.619932 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:11.620078 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:11.620505 (+ 427us) server_negotiation.cc:299] Negotiation successful
0619 06:45:11.620731 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 06:45:11.621202 24435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:11.606308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38019 (local address 127.16.79.65:55779)
0619 06:45:11.607531 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:11.607555 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:11.607707 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:11.608247 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:11.608254 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:11.608846 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:11.609385 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:11.609396 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:11.610697 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:11.610705 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:11.612378 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:11.612390 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:11.613102 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:11.613108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:11.613293 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:11.614075 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:11.614092 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:11.615238 (+ 1146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:11.618650 (+ 3412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:11.618655 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:11.618659 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:11.619030 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:11.619418 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:11.619424 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:11.619428 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:11.619551 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:11.620043 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:11.620048 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:11.620266 (+ 218us) client_negotiation.cc:769] Sending connection context
0619 06:45:11.620423 (+ 157us) client_negotiation.cc:240] Negotiation successful
0619 06:45:11.620743 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1064,"thread_start_us":851,"threads_started":1}
I20260619 06:45:11.627990 24337 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:35559
I20260619 06:45:11.628072 24472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:35559 every 8 connection(s)
I20260619 06:45:11.629261 24337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:45:11.631353 24473 heartbeater.cc:347] Connected to a master server at 127.16.79.126:38019
I20260619 06:45:11.631595 24473 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:11.632373 24473 heartbeater.cc:511] Master 127.16.79.126:38019 requested a full tablet report, sending...
I20260619 06:45:11.634419 24265 ts_manager.cc:194] Registered new tserver with Master: 9552fb76d538450791575265a9240c9e (127.16.79.65:35559)
I20260619 06:45:11.635838 24265 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:55811
I20260619 06:45:11.638470 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 24337
I20260619 06:45:11.638655 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:45:11.639300 16701 external_mini_cluster.cc:1428] Setting key 6dac617e92c8758bf5c48b85495dac6d
WARNING: no policy specified for oryx/127.16.79.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.79.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.66.keytab.
Entry for principal oryx/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.66.keytab.
I20260619 06:45:11.684736 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:38019
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.66.keytab
--principal=oryx/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:43455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:12.015407 24477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:12.015728 24477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:12.015898 24477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:12.023173 24477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:12.023330 24477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:12.023471 24477 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:45:12.023607 24477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:45:12.033990 24477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:43455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.66.keytab
--principal=oryx/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.79.126:38019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:12.035356 24477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:12.036772 24477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:12.049862 24483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:12.049912 24482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:12.049863 24485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:12.050839 24477 server_base.cc:1034] running on GCE node
I20260619 06:45:12.051534 24477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:12.052552 24477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:12.053784 24477 hybrid_clock.cc:648] HybridClock initialized: now 1781851512053715 us; error 42 us; skew 500 ppm
I20260619 06:45:12.054052 24477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:12 dist-test-slave-5w2x krb5kdc[24220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851512, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:12.057361 24477 init.cc:377] Logged in from keytab as oryx/127.16.79.66@KRBTEST.COM (short username oryx)
I20260619 06:45:12.058871 24477 webserver.cc:460] Webserver started at http://127.16.79.66:43735/ using document root <none> and password file <none>
I20260619 06:45:12.059602 24477 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:12.059695 24477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:12.060045 24477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:12.062986 24477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "a8a35995e3cc44d58a1cd2228c740fa3"
format_stamp: "Formatted at 2026-06-19 06:45:12 on dist-test-slave-5w2x"
server_key: "2fb281f233bd99e29c4aad657c48c70f"
server_key_iv: "e6237f0fa9cbf108ece17599130c5311"
server_key_version: "encryptionkey@0"
I20260619 06:45:12.063828 24477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "a8a35995e3cc44d58a1cd2228c740fa3"
format_stamp: "Formatted at 2026-06-19 06:45:12 on dist-test-slave-5w2x"
server_key: "2fb281f233bd99e29c4aad657c48c70f"
server_key_iv: "e6237f0fa9cbf108ece17599130c5311"
server_key_version: "encryptionkey@0"
I20260619 06:45:12.069095 24477 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 06:45:12.072629 24492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:12.074023 24477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260619 06:45:12.074198 24477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "a8a35995e3cc44d58a1cd2228c740fa3"
format_stamp: "Formatted at 2026-06-19 06:45:12 on dist-test-slave-5w2x"
server_key: "2fb281f233bd99e29c4aad657c48c70f"
server_key_iv: "e6237f0fa9cbf108ece17599130c5311"
server_key_version: "encryptionkey@0"
I20260619 06:45:12.074429 24477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:12.114171 24477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:12.118479 24477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:12.118760 24477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:12.121681 24477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:12.121788 24477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:12.121950 24477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:12.122025 24477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 06:45:12 dist-test-slave-5w2x krb5kdc[24220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851512, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM
I20260619 06:45:12.146474 24501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:12.122262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:36531 (local address 127.16.79.126:38019)
0619 06:45:12.122955 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:12.122963 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:12.124161 (+ 1198us) server_negotiation.cc:407] Connection header received
0619 06:45:12.126838 (+ 2677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:12.126847 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:12.126964 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:12.127106 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:12.129550 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.130652 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.132931 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.133658 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.133796 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:12.138586 (+ 4790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:12.138596 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:12.138604 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:12.138658 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:12.142604 (+ 3946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:12.143716 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:12.143722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:12.143728 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:12.143871 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:12.144520 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:12.144523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:12.144526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:12.144789 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:12.144938 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:12.145866 (+ 928us) server_negotiation.cc:299] Negotiation successful
0619 06:45:12.146099 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":449,"threads_started":1}
I20260619 06:45:12.147847 24502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:12.123020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38019 (local address 127.16.79.66:36531)
0619 06:45:12.123912 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:12.123969 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:12.126404 (+ 2435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:12.127421 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:12.127438 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:12.128334 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:12.129317 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:12.129370 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.130822 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.130828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:12.132762 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:12.132780 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.133916 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.133927 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:12.134169 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:12.134947 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:12.134994 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:12.138259 (+ 3265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:12.142789 (+ 4530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:12.142799 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:12.142817 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:12.143493 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:12.143980 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:12.143986 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:12.143990 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:12.144359 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:12.144915 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:12.144923 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:12.145475 (+ 552us) client_negotiation.cc:769] Sending connection context
0619 06:45:12.145766 (+ 291us) client_negotiation.cc:240] Negotiation successful
0619 06:45:12.146353 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":368,"threads_started":1}
I20260619 06:45:12.170090 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:12.154172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38019 (local address 127.16.79.66:57495)
0619 06:45:12.155062 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:12.155087 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:12.155263 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:12.155636 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:12.155644 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:12.156396 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:12.156920 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:12.156937 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.157960 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.157969 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:12.160209 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:12.160229 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.161002 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.161009 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:12.161254 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:12.162298 (+ 1044us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:12.162328 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:12.163288 (+ 960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:12.167131 (+ 3843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:12.167136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:12.167140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:12.167540 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:12.167904 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:12.167910 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:12.167913 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:12.168036 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:12.168774 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:12.168778 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:12.168947 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 06:45:12.169135 (+ 188us) client_negotiation.cc:240] Negotiation successful
0619 06:45:12.169488 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":426,"threads_started":1}
I20260619 06:45:12.171083 24501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:12.153914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:57495 (local address 127.16.79.126:38019)
0619 06:45:12.154102 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:12.154108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:12.155192 (+ 1084us) server_negotiation.cc:407] Connection header received
0619 06:45:12.155386 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:12.155390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:12.155488 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:12.155575 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:12.157118 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.157810 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.160372 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.160814 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.160925 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:12.163496 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:12.163503 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:12.163508 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:12.163550 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:12.166981 (+ 3431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:12.167659 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:12.167666 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:12.167670 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:12.167799 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:12.168170 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:12.168176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:12.168182 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:12.168531 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:12.168700 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:12.170607 (+ 1907us) server_negotiation.cc:299] Negotiation successful
0619 06:45:12.170777 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 06:45:12.174564 24477 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:36045
I20260619 06:45:12.174607 24612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:36045 every 8 connection(s)
I20260619 06:45:12.175843 24477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:45:12.177242 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 24477
I20260619 06:45:12.177413 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:45:12.177977 16701 external_mini_cluster.cc:1428] Setting key 0598abd81997b3c8b660874f5662ed25
I20260619 06:45:12.177977 24613 heartbeater.cc:347] Connected to a master server at 127.16.79.126:38019
I20260619 06:45:12.179265 24613 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:12.180070 24613 heartbeater.cc:511] Master 127.16.79.126:38019 requested a full tablet report, sending...
I20260619 06:45:12.181401 24265 ts_manager.cc:194] Registered new tserver with Master: a8a35995e3cc44d58a1cd2228c740fa3 (127.16.79.66:36045)
I20260619 06:45:12.182206 24265 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:36531
WARNING: no policy specified for oryx/127.16.79.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.79.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.67.keytab.
Entry for principal oryx/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.67.keytab.
I20260619 06:45:12.222781 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:38019
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.67.keytab
--principal=oryx/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:43455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:12.552619 24617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:12.552963 24617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:12.553143 24617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:12.561177 24617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:12.561337 24617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:12.561498 24617 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 06:45:12.561653 24617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:45:12.572294 24617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:43455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.67.keytab
--principal=oryx/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.16.79.126:38019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:12.573609 24617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:12.574930 24617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:12.587890 24622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:12.587971 24625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:12.587946 24623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:12.588399 24617 server_base.cc:1034] running on GCE node
I20260619 06:45:12.589212 24617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:12.590131 24617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:12.591379 24617 hybrid_clock.cc:648] HybridClock initialized: now 1781851512591301 us; error 56 us; skew 500 ppm
I20260619 06:45:12.591691 24617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:12 dist-test-slave-5w2x krb5kdc[24220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851512, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:12.594909 24617 init.cc:377] Logged in from keytab as oryx/127.16.79.67@KRBTEST.COM (short username oryx)
I20260619 06:45:12.596511 24617 webserver.cc:460] Webserver started at http://127.16.79.67:42207/ using document root <none> and password file <none>
I20260619 06:45:12.597211 24617 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:12.597339 24617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:12.597658 24617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:12.600818 24617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "c9afac029e4e456b85384f7adbdb84d0"
format_stamp: "Formatted at 2026-06-19 06:45:12 on dist-test-slave-5w2x"
server_key: "391a6d9675b4c6f3969cb76e0bb5e581"
server_key_iv: "0f137872c4dbce64a68a85cab305b78d"
server_key_version: "encryptionkey@0"
I20260619 06:45:12.601598 24617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "c9afac029e4e456b85384f7adbdb84d0"
format_stamp: "Formatted at 2026-06-19 06:45:12 on dist-test-slave-5w2x"
server_key: "391a6d9675b4c6f3969cb76e0bb5e581"
server_key_iv: "0f137872c4dbce64a68a85cab305b78d"
server_key_version: "encryptionkey@0"
I20260619 06:45:12.606712 24617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260619 06:45:12.610211 24632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:12.611523 24617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:45:12.611698 24617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "c9afac029e4e456b85384f7adbdb84d0"
format_stamp: "Formatted at 2026-06-19 06:45:12 on dist-test-slave-5w2x"
server_key: "391a6d9675b4c6f3969cb76e0bb5e581"
server_key_iv: "0f137872c4dbce64a68a85cab305b78d"
server_key_version: "encryptionkey@0"
I20260619 06:45:12.611903 24617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:12.632282 24617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:12.636552 24617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:12.636794 24617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:12.638396 24473 heartbeater.cc:503] Master 127.16.79.126:38019 was elected leader, sending a full tablet report...
I20260619 06:45:12.639583 24617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:12.639679 24617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:12.639838 24617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:12.639961 24617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Jun 19 06:45:12 dist-test-slave-5w2x krb5kdc[24220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851512, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM
I20260619 06:45:12.663887 24501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:12.640113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:47571 (local address 127.16.79.126:38019)
0619 06:45:12.640382 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:12.640389 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:12.641713 (+ 1324us) server_negotiation.cc:407] Connection header received
0619 06:45:12.643901 (+ 2188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:12.643906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:12.643992 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:12.644090 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:12.646781 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.647774 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.650609 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.651347 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.651508 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:12.656329 (+ 4821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:12.656339 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:12.656342 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:12.656392 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:12.659796 (+ 3404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:12.661044 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:12.661048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:12.661050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:12.661174 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:12.661774 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:12.661779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:12.661782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:12.662156 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:12.662316 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:12.663316 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0619 06:45:12.663524 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260619 06:45:12.665412 24641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:12.640797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38019 (local address 127.16.79.67:47571)
0619 06:45:12.641572 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:12.641608 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:12.643645 (+ 2037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:12.644378 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:12.644390 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:12.645780 (+ 1390us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:12.646608 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:12.646626 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.648002 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.648015 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:12.650414 (+ 2399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:12.650444 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.651614 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.651622 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:12.651841 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:12.652640 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:12.652669 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:12.656049 (+ 3380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:12.660001 (+ 3952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:12.660016 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:12.660040 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:12.660901 (+ 861us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:12.661302 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:12.661310 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:12.661315 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:12.661653 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:12.662377 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:12.662383 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:12.662940 (+ 557us) client_negotiation.cc:769] Sending connection context
0619 06:45:12.663208 (+ 268us) client_negotiation.cc:240] Negotiation successful
0619 06:45:12.663750 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":346,"threads_started":1}
I20260619 06:45:12.689155 24617 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:38139
I20260619 06:45:12.690821 24617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:45:12.689126 24751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:38139 every 8 connection(s)
I20260619 06:45:12.693045 24752 heartbeater.cc:347] Connected to a master server at 127.16.79.126:38019
I20260619 06:45:12.693102 24501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:12.678903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:44349 (local address 127.16.79.126:38019)
0619 06:45:12.679111 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:12.679119 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:12.679838 (+ 719us) server_negotiation.cc:407] Connection header received
0619 06:45:12.680040 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:12.680044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:12.680112 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:12.680298 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:12.681638 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.682679 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.684779 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.685234 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.685358 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:12.687135 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:12.687141 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:12.687144 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:12.687173 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:12.690265 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:12.690908 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:12.690913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:12.690916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:12.691076 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:12.691481 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:12.691487 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:12.691491 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:12.691851 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:12.691998 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:12.692529 (+ 531us) server_negotiation.cc:299] Negotiation successful
0619 06:45:12.692740 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 06:45:12.693671 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 24617
I20260619 06:45:12.693831 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:45:12.694276 16701 external_mini_cluster.cc:1428] Setting key 133047bc5f9eecd9bcb69d44219fcfab
I20260619 06:45:12.693148 24734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:12.678902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38019 (local address 127.16.79.67:44349)
0619 06:45:12.679712 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:12.679738 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:12.679920 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:12.680332 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:12.680337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:12.681014 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:12.681475 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:12.681494 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.682814 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.682820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:12.684630 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:12.684642 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.685426 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.685433 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:12.685577 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:12.686252 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:12.686268 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:12.686953 (+ 685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:12.690389 (+ 3436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:12.690393 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:12.690396 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:12.690787 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:12.691183 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:12.691186 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:12.691188 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:12.691303 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:12.692058 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:12.692063 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:12.692287 (+ 224us) client_negotiation.cc:769] Sending connection context
0619 06:45:12.692425 (+ 138us) client_negotiation.cc:240] Negotiation successful
0619 06:45:12.692701 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":434,"threads_started":1}
I20260619 06:45:12.694977 24752 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:12.696170 24752 heartbeater.cc:511] Master 127.16.79.126:38019 requested a full tablet report, sending...
I20260619 06:45:12.697748 24265 ts_manager.cc:194] Registered new tserver with Master: c9afac029e4e456b85384f7adbdb84d0 (127.16.79.67:38139)
I20260619 06:45:12.698830 24265 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:47571
I20260619 06:45:12.709667 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 06:45:12 dist-test-slave-5w2x krb5kdc[24220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851512, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 06:45:12 dist-test-slave-5w2x krb5kdc[24220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:12 dist-test-slave-5w2x krb5kdc[24220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:13.184561 24613 heartbeater.cc:503] Master 127.16.79.126:38019 was elected leader, sending a full tablet report...
I20260619 06:45:13.701298 24752 heartbeater.cc:503] Master 127.16.79.126:38019 was elected leader, sending a full tablet report...
W20260619 06:45:13.804919 24760 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.052s user 0.006s sys 0.008s
W20260619 06:45:13.806146 24501 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:12.746821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46136 (local address 127.16.79.126:38019)
0619 06:45:12.747052 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:12.747060 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:12.747311 (+ 251us) server_negotiation.cc:407] Connection header received
0619 06:45:12.747520 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:12.747529 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:12.747589 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:12.747697 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:12.749035 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.749910 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.751198 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:12.751596 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:12.751730 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:13.805840 (+1054110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46136: BlockingRecv error: recv got EOF from 127.0.0.1:46136 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Jun 19 06:45:13 dist-test-slave-5w2x krb5kdc[24220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851512, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM
I20260619 06:45:13.827013 24501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:13.809488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46142 (local address 127.16.79.126:38019)
0619 06:45:13.809646 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:13.809651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:13.810010 (+ 359us) server_negotiation.cc:407] Connection header received
0619 06:45:13.810169 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:13.810172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:13.810232 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:13.810309 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:13.811435 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:13.812214 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:13.813422 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:13.813863 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:13.813951 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:13.822141 (+ 8190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:13.822147 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:13.822150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:13.822185 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:13.824769 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:13.825360 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:13.825363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:13.825366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:13.825480 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:13.825861 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:13.825865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:13.825867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:13.826207 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:13.826418 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:13.826616 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 06:45:13.826785 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 06:45:13.830742 24265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:45:13.837634 24265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:45:13.869256 24776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:13.851328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58000 (local address 127.16.79.65:35559)
0619 06:45:13.860404 (+ 9076us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:13.860417 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:13.860452 (+ 35us) server_negotiation.cc:407] Connection header received
0619 06:45:13.860572 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:13.860582 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:13.860875 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:13.861103 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:13.862310 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:13.864419 (+ 2109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:13.867117 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:13.868059 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:13.868297 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:13.868521 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:13.868797 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 06:45:13.868959 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8884,"thread_start_us":8702,"threads_started":1}
I20260619 06:45:13.869328 24774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:13.850293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:35559 (local address 127.0.0.1:58000)
0619 06:45:13.859408 (+ 9115us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:13.859432 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:13.859575 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:13.861400 (+ 1825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:13.861408 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:13.861444 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:13.862051 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:13.862068 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:13.864635 (+ 2567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:13.864645 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:13.866922 (+ 2277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:13.866936 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:13.868377 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:13.868385 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:13.868567 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:13.868617 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 06:45:13.868747 (+ 130us) client_negotiation.cc:240] Negotiation successful
0619 06:45:13.868933 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8943,"thread_start_us":14589,"threads_started":1}
I20260619 06:45:13.871882 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:13.849962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:36045 (local address 127.0.0.1:40424)
0619 06:45:13.857612 (+ 7650us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:13.857666 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:13.857879 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:13.860759 (+ 2880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:13.860771 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:13.860818 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:13.863058 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:13.863076 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:13.865580 (+ 2504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:13.865585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:13.868313 (+ 2728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:13.868328 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:13.871086 (+ 2758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:13.871093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:13.871276 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:13.871311 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 06:45:13.871410 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 06:45:13.871544 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7374,"mutex_wait_us":2,"thread_start_us":7170,"threads_started":1}
I20260619 06:45:13.871976 24772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:13.850129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40424 (local address 127.16.79.66:36045)
0619 06:45:13.859523 (+ 9394us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:13.859537 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:13.859582 (+ 45us) server_negotiation.cc:407] Connection header received
0619 06:45:13.859728 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:13.859740 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:13.860097 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:13.860396 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:13.863347 (+ 2951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:13.865411 (+ 2064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:13.868501 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:13.870868 (+ 2367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:13.871017 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:13.871237 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:13.871483 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 06:45:13.871654 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9137,"thread_start_us":9528,"threads_started":1}
I20260619 06:45:13.873544 24403 tablet_service.cc:1467] Processing CreateTablet for tablet 20285c0b3d004653913fca66ab11efb4 (DEFAULT_TABLE table=test-table [id=1761e98ac64843e0b71fc231590877c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:13.875056 24773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:13.850936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:38139 (local address 127.0.0.1:37886)
0619 06:45:13.865228 (+ 14292us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:13.865248 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:13.865373 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:13.866157 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:13.866162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:13.866187 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:13.866664 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:13.866682 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:13.868860 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:13.868866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:13.871201 (+ 2335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:13.871217 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:13.873149 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:13.873156 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:13.874351 (+ 1195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:13.874419 (+ 68us) client_negotiation.cc:769] Sending connection context
0619 06:45:13.874521 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 06:45:13.874682 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14107,"thread_start_us":8171,"threads_started":1}
I20260619 06:45:13.875324 24403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20285c0b3d004653913fca66ab11efb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:13.876173 24775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:13.851219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37886 (local address 127.16.79.67:38139)
0619 06:45:13.864466 (+ 13247us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:13.864475 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:13.865526 (+ 1051us) server_negotiation.cc:407] Connection header received
0619 06:45:13.865636 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:13.865647 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:13.865918 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:13.866879 (+ 961us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:13.867095 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:13.868701 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:13.871341 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:13.872923 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:13.873099 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:13.873309 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:13.875667 (+ 2358us) server_negotiation.cc:299] Negotiation successful
0619 06:45:13.875847 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13078,"thread_start_us":13253,"threads_started":1}
I20260619 06:45:13.875878 24543 tablet_service.cc:1467] Processing CreateTablet for tablet 20285c0b3d004653913fca66ab11efb4 (DEFAULT_TABLE table=test-table [id=1761e98ac64843e0b71fc231590877c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:13.877436 24543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20285c0b3d004653913fca66ab11efb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:13.879592 24682 tablet_service.cc:1467] Processing CreateTablet for tablet 20285c0b3d004653913fca66ab11efb4 (DEFAULT_TABLE table=test-table [id=1761e98ac64843e0b71fc231590877c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:13.880903 24682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20285c0b3d004653913fca66ab11efb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:13.889323 24777 tablet_bootstrap.cc:492] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3: Bootstrap starting.
I20260619 06:45:13.891249 24778 tablet_bootstrap.cc:492] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e: Bootstrap starting.
I20260619 06:45:13.893007 24777 tablet_bootstrap.cc:654] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:13.894289 24777 log.cc:824] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:13.894485 24779 tablet_bootstrap.cc:492] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0: Bootstrap starting.
I20260619 06:45:13.894654 24778 tablet_bootstrap.cc:654] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:13.895803 24778 log.cc:824] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:13.897979 24779 tablet_bootstrap.cc:654] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:13.899004 24779 log.cc:824] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:13.899101 24777 tablet_bootstrap.cc:492] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3: No bootstrap required, opened a new log
I20260619 06:45:13.899423 24777 ts_tablet_manager.cc:1397] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260619 06:45:13.901927 24778 tablet_bootstrap.cc:492] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e: No bootstrap required, opened a new log
I20260619 06:45:13.902096 24779 tablet_bootstrap.cc:492] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0: No bootstrap required, opened a new log
I20260619 06:45:13.902232 24778 ts_tablet_manager.cc:1397] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260619 06:45:13.902420 24779 ts_tablet_manager.cc:1397] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260619 06:45:13.910790 24778 raft_consensus.cc:348] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:13.910745 24777 raft_consensus.cc:348] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:13.911177 24778 raft_consensus.cc:374] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:13.911186 24777 raft_consensus.cc:374] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:13.911293 24778 raft_consensus.cc:729] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9552fb76d538450791575265a9240c9e, State: Initialized, Role: FOLLOWER
I20260619 06:45:13.911342 24777 raft_consensus.cc:729] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8a35995e3cc44d58a1cd2228c740fa3, State: Initialized, Role: FOLLOWER
I20260619 06:45:13.911937 24778 consensus_queue.cc:260] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:13.911966 24777 consensus_queue.cc:260] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:13.912935 24779 raft_consensus.cc:348] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:13.913431 24779 raft_consensus.cc:374] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:13.913545 24779 raft_consensus.cc:729] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9afac029e4e456b85384f7adbdb84d0, State: Initialized, Role: FOLLOWER
I20260619 06:45:13.914315 24779 consensus_queue.cc:260] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:13.915117 24778 ts_tablet_manager.cc:1428] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
I20260619 06:45:13.916518 24779 ts_tablet_manager.cc:1428] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260619 06:45:13.918963 24777 ts_tablet_manager.cc:1428] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3: Time spent starting tablet: real 0.019s user 0.009s sys 0.006s
W20260619 06:45:13.928540 24614 tablet.cc:2369] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:45:13.943449 24753 tablet.cc:2369] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:45:14.132849 24474 tablet.cc:2369] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:45:14.270979 24783 raft_consensus.cc:482] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:45:14.271353 24783 raft_consensus.cc:504] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.273093 24783 leader_election.cc:288] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8a35995e3cc44d58a1cd2228c740fa3 (127.16.79.66:36045), c9afac029e4e456b85384f7adbdb84d0 (127.16.79.67:38139)
I20260619 06:45:14.279742 24775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:14.273627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:42905 (local address 127.16.79.67:38139)
0619 06:45:14.274032 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:14.274041 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:14.274323 (+ 282us) server_negotiation.cc:407] Connection header received
0619 06:45:14.274560 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:14.274566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:14.274635 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:14.274725 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:14.275289 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.275988 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.277923 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.278776 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.278957 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:14.279033 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:14.279356 (+ 323us) server_negotiation.cc:299] Negotiation successful
0619 06:45:14.279482 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224}
I20260619 06:45:14.279878 24772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:14.273567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:36957 (local address 127.16.79.66:36045)
0619 06:45:14.273810 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:14.273818 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:14.274618 (+ 800us) server_negotiation.cc:407] Connection header received
0619 06:45:14.274827 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:14.274836 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:14.274894 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:14.274984 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:14.275490 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.276240 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.278011 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.278942 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.279121 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:14.279208 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:14.279504 (+ 296us) server_negotiation.cc:299] Negotiation successful
0619 06:45:14.279628 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 06:45:14.279929 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:14.273412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:38139 (local address 127.16.79.65:42905)
0619 06:45:14.274210 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:14.274227 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:14.274416 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:14.274754 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:14.274758 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:14.274783 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:14.275147 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:14.275156 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.276129 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.276159 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:14.277767 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:14.277779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.278985 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.278992 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:14.279170 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:14.279195 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 06:45:14.279323 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 06:45:14.279467 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":464,"threads_started":1}
I20260619 06:45:14.279989 24787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:14.273536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:36045 (local address 127.16.79.65:36957)
0619 06:45:14.274507 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:14.274525 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:14.274697 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:14.274996 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:14.275000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:14.275028 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:14.275351 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:14.275359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.276364 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.276369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:14.277878 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:14.277890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.279165 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.279172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:14.279355 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:14.279385 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 06:45:14.279480 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 06:45:14.279617 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":860,"thread_start_us":691,"threads_started":1}
I20260619 06:45:14.281019 24563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20285c0b3d004653913fca66ab11efb4" candidate_uuid: "9552fb76d538450791575265a9240c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" is_pre_election: true
I20260619 06:45:14.281020 24702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20285c0b3d004653913fca66ab11efb4" candidate_uuid: "9552fb76d538450791575265a9240c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9afac029e4e456b85384f7adbdb84d0" is_pre_election: true
I20260619 06:45:14.281520 24563 raft_consensus.cc:2454] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9552fb76d538450791575265a9240c9e in term 0.
I20260619 06:45:14.281527 24702 raft_consensus.cc:2454] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9552fb76d538450791575265a9240c9e in term 0.
I20260619 06:45:14.282124 24356 leader_election.cc:302] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9552fb76d538450791575265a9240c9e, c9afac029e4e456b85384f7adbdb84d0; no voters:
I20260619 06:45:14.282526 24783 raft_consensus.cc:2781] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:45:14.282667 24783 raft_consensus.cc:482] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:45:14.282774 24783 raft_consensus.cc:3037] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:14.285023 24783 raft_consensus.cc:504] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.285655 24783 leader_election.cc:288] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [CANDIDATE]: Term 1 election: Requested vote from peers a8a35995e3cc44d58a1cd2228c740fa3 (127.16.79.66:36045), c9afac029e4e456b85384f7adbdb84d0 (127.16.79.67:38139)
I20260619 06:45:14.286139 24563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20285c0b3d004653913fca66ab11efb4" candidate_uuid: "9552fb76d538450791575265a9240c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8a35995e3cc44d58a1cd2228c740fa3"
I20260619 06:45:14.286170 24702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20285c0b3d004653913fca66ab11efb4" candidate_uuid: "9552fb76d538450791575265a9240c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9afac029e4e456b85384f7adbdb84d0"
I20260619 06:45:14.286365 24563 raft_consensus.cc:3037] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:14.286365 24702 raft_consensus.cc:3037] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:14.289662 24702 raft_consensus.cc:2454] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9552fb76d538450791575265a9240c9e in term 1.
I20260619 06:45:14.289662 24563 raft_consensus.cc:2454] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9552fb76d538450791575265a9240c9e in term 1.
I20260619 06:45:14.290179 24355 leader_election.cc:302] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9552fb76d538450791575265a9240c9e, a8a35995e3cc44d58a1cd2228c740fa3, c9afac029e4e456b85384f7adbdb84d0; no voters:
I20260619 06:45:14.290503 24783 raft_consensus.cc:2781] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:45:14.290900 24783 raft_consensus.cc:686] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [term 1 LEADER]: Becoming Leader. State: Replica: 9552fb76d538450791575265a9240c9e, State: Running, Role: LEADER
I20260619 06:45:14.291402 24783 consensus_queue.cc:237] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.296382 24264 catalog_manager.cc:5495] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9552fb76d538450791575265a9240c9e (127.16.79.65). New cstate: current_term: 1 leader_uuid: "9552fb76d538450791575265a9240c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } health_report { overall_health: UNKNOWN } } }
I20260619 06:45:14.323000 24776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:14.317917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58008 (local address 127.16.79.65:35559)
0619 06:45:14.318087 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:14.318092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:14.318109 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:45:14.318295 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:14.318299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:14.318376 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:14.318505 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:45:14.318841 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.319513 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.320915 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.321354 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.321459 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:14.321887 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:45:14.322458 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:45:14.322567 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:14.322697 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 06:45:14.322796 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 06:45:14.331030 24563 raft_consensus.cc:1261] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Refusing update from remote peer 9552fb76d538450791575265a9240c9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:45:14.331027 24702 raft_consensus.cc:1261] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Refusing update from remote peer 9552fb76d538450791575265a9240c9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:45:14.331951 24788 consensus_queue.cc:1035] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:45:14.332401 24783 consensus_queue.cc:1035] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:45:14.343250 24791 mvcc.cc:205] Tried to move back new op lower bound from 7298463802690969600 to 7298463802541588480. Current Snapshot: MvccSnapshot[applied={T|T < 7298463802690969600}]
I20260619 06:45:14.362690 24265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:45:14.362941 24265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:45:14.366185 24265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b: Sending DeleteTablet for 3 replicas of tablet 20285c0b3d004653913fca66ab11efb4
I20260619 06:45:14.367297 24543 tablet_service.cc:1514] Processing DeleteTablet for tablet 20285c0b3d004653913fca66ab11efb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:14 UTC) from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:40424
I20260619 06:45:14.367537 24682 tablet_service.cc:1514] Processing DeleteTablet for tablet 20285c0b3d004653913fca66ab11efb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:14 UTC) from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:37886
I20260619 06:45:14.367990 24403 tablet_service.cc:1514] Processing DeleteTablet for tablet 20285c0b3d004653913fca66ab11efb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:14 UTC) from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:58000
I20260619 06:45:14.369020 24265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:45:14.370177 24265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:45:14.377902 24803 tablet_replica.cc:330] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0: stopping tablet replica
I20260619 06:45:14.378623 24803 raft_consensus.cc:2229] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:45:14.379812 24402 tablet_service.cc:1467] Processing CreateTablet for tablet e0b10272864546eda9657257374e3048 (DEFAULT_TABLE table=test-table [id=ff8ba6d402c24c07bcb96825b9e09a73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:14.379864 24542 tablet_service.cc:1467] Processing CreateTablet for tablet e0b10272864546eda9657257374e3048 (DEFAULT_TABLE table=test-table [id=ff8ba6d402c24c07bcb96825b9e09a73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:14.380439 24402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0b10272864546eda9657257374e3048. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:14.380578 24542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0b10272864546eda9657257374e3048. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:14.381160 24802 tablet_replica.cc:330] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3: stopping tablet replica
I20260619 06:45:14.381569 24802 raft_consensus.cc:2229] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:45:14.382068 24802 raft_consensus.cc:2258] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:14.382827 24804 tablet_replica.cc:330] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e: stopping tablet replica
I20260619 06:45:14.383234 24804 raft_consensus.cc:2229] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:45:14.383927 24804 raft_consensus.cc:2258] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:14.387089 24777 tablet_bootstrap.cc:492] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3: Bootstrap starting.
I20260619 06:45:14.387741 24682 tablet_service.cc:1467] Processing CreateTablet for tablet e0b10272864546eda9657257374e3048 (DEFAULT_TABLE table=test-table [id=ff8ba6d402c24c07bcb96825b9e09a73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:14.387846 24802 ts_tablet_manager.cc:1905] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:14.388212 24682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0b10272864546eda9657257374e3048. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:14.389230 24777 tablet_bootstrap.cc:654] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:14.391285 24804 ts_tablet_manager.cc:1905] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:14.394443 24778 tablet_bootstrap.cc:492] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e: Bootstrap starting.
I20260619 06:45:14.396252 24778 tablet_bootstrap.cc:654] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:14.397511 24803 raft_consensus.cc:2258] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:14.399904 24803 ts_tablet_manager.cc:1905] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:14.401970 24802 ts_tablet_manager.cc:1918] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:45:14.402196 24802 log.cc:1196] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/20285c0b3d004653913fca66ab11efb4
I20260619 06:45:14.402657 24777 tablet_bootstrap.cc:492] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3: No bootstrap required, opened a new log
I20260619 06:45:14.402835 24802 ts_tablet_manager.cc:1939] T 20285c0b3d004653913fca66ab11efb4 P a8a35995e3cc44d58a1cd2228c740fa3: Deleting consensus metadata
I20260619 06:45:14.402886 24777 ts_tablet_manager.cc:1397] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.003s
I20260619 06:45:14.403708 24777 raft_consensus.cc:348] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.403911 24777 raft_consensus.cc:374] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:14.404011 24777 raft_consensus.cc:729] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8a35995e3cc44d58a1cd2228c740fa3, State: Initialized, Role: FOLLOWER
I20260619 06:45:14.404332 24777 consensus_queue.cc:260] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.404821 24251 catalog_manager.cc:4841] TS a8a35995e3cc44d58a1cd2228c740fa3 (127.16.79.66:36045): tablet 20285c0b3d004653913fca66ab11efb4 (table test-table [id=1761e98ac64843e0b71fc231590877c9]) successfully deleted
I20260619 06:45:14.405433 24779 tablet_bootstrap.cc:492] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0: Bootstrap starting.
I20260619 06:45:14.406689 24803 ts_tablet_manager.cc:1918] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:45:14.406821 24803 log.cc:1196] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/20285c0b3d004653913fca66ab11efb4
I20260619 06:45:14.407117 24779 tablet_bootstrap.cc:654] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:14.407409 24803 ts_tablet_manager.cc:1939] T 20285c0b3d004653913fca66ab11efb4 P c9afac029e4e456b85384f7adbdb84d0: Deleting consensus metadata
I20260619 06:45:14.408588 24777 ts_tablet_manager.cc:1428] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260619 06:45:14.409317 24252 catalog_manager.cc:4841] TS c9afac029e4e456b85384f7adbdb84d0 (127.16.79.67:38139): tablet 20285c0b3d004653913fca66ab11efb4 (table test-table [id=1761e98ac64843e0b71fc231590877c9]) successfully deleted
I20260619 06:45:14.409914 24804 ts_tablet_manager.cc:1918] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:45:14.410164 24804 log.cc:1196] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/20285c0b3d004653913fca66ab11efb4
I20260619 06:45:14.410755 24804 ts_tablet_manager.cc:1939] T 20285c0b3d004653913fca66ab11efb4 P 9552fb76d538450791575265a9240c9e: Deleting consensus metadata
I20260619 06:45:14.412504 24250 catalog_manager.cc:4841] TS 9552fb76d538450791575265a9240c9e (127.16.79.65:35559): tablet 20285c0b3d004653913fca66ab11efb4 (table test-table [id=1761e98ac64843e0b71fc231590877c9]) successfully deleted
I20260619 06:45:14.413148 24779 tablet_bootstrap.cc:492] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0: No bootstrap required, opened a new log
I20260619 06:45:14.413352 24779 ts_tablet_manager.cc:1397] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.005s
I20260619 06:45:14.414150 24778 tablet_bootstrap.cc:492] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e: No bootstrap required, opened a new log
I20260619 06:45:14.414213 24779 raft_consensus.cc:348] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.414342 24778 ts_tablet_manager.cc:1397] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.005s
I20260619 06:45:14.414430 24779 raft_consensus.cc:374] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:14.414525 24779 raft_consensus.cc:729] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9afac029e4e456b85384f7adbdb84d0, State: Initialized, Role: FOLLOWER
I20260619 06:45:14.414830 24779 consensus_queue.cc:260] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.415292 24778 raft_consensus.cc:348] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.415410 24779 ts_tablet_manager.cc:1428] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 06:45:14.415549 24778 raft_consensus.cc:374] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:14.415680 24778 raft_consensus.cc:729] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9552fb76d538450791575265a9240c9e, State: Initialized, Role: FOLLOWER
I20260619 06:45:14.415978 24778 consensus_queue.cc:260] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.416615 24778 ts_tablet_manager.cc:1428] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 06:45:14.586621 24800 raft_consensus.cc:482] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:45:14.586843 24800 raft_consensus.cc:504] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.587634 24800 leader_election.cc:288] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8a35995e3cc44d58a1cd2228c740fa3 (127.16.79.66:36045), c9afac029e4e456b85384f7adbdb84d0 (127.16.79.67:38139)
I20260619 06:45:14.588230 24563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0b10272864546eda9657257374e3048" candidate_uuid: "9552fb76d538450791575265a9240c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" is_pre_election: true
I20260619 06:45:14.588213 24702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0b10272864546eda9657257374e3048" candidate_uuid: "9552fb76d538450791575265a9240c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9afac029e4e456b85384f7adbdb84d0" is_pre_election: true
I20260619 06:45:14.588474 24563 raft_consensus.cc:2454] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9552fb76d538450791575265a9240c9e in term 0.
I20260619 06:45:14.588476 24702 raft_consensus.cc:2454] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9552fb76d538450791575265a9240c9e in term 0.
I20260619 06:45:14.588894 24356 leader_election.cc:302] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9552fb76d538450791575265a9240c9e, c9afac029e4e456b85384f7adbdb84d0; no voters:
I20260619 06:45:14.589174 24800 raft_consensus.cc:2781] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:45:14.589322 24800 raft_consensus.cc:482] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:45:14.589432 24800 raft_consensus.cc:3037] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:14.591629 24800 raft_consensus.cc:504] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.592233 24800 leader_election.cc:288] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [CANDIDATE]: Term 1 election: Requested vote from peers a8a35995e3cc44d58a1cd2228c740fa3 (127.16.79.66:36045), c9afac029e4e456b85384f7adbdb84d0 (127.16.79.67:38139)
I20260619 06:45:14.592706 24563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0b10272864546eda9657257374e3048" candidate_uuid: "9552fb76d538450791575265a9240c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8a35995e3cc44d58a1cd2228c740fa3"
I20260619 06:45:14.592797 24702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0b10272864546eda9657257374e3048" candidate_uuid: "9552fb76d538450791575265a9240c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9afac029e4e456b85384f7adbdb84d0"
I20260619 06:45:14.592906 24563 raft_consensus.cc:3037] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:14.593032 24702 raft_consensus.cc:3037] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:14.594898 24563 raft_consensus.cc:2454] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9552fb76d538450791575265a9240c9e in term 1.
I20260619 06:45:14.595345 24355 leader_election.cc:302] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9552fb76d538450791575265a9240c9e, a8a35995e3cc44d58a1cd2228c740fa3; no voters:
I20260619 06:45:14.595623 24800 raft_consensus.cc:2781] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:45:14.595790 24800 raft_consensus.cc:686] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [term 1 LEADER]: Becoming Leader. State: Replica: 9552fb76d538450791575265a9240c9e, State: Running, Role: LEADER
I20260619 06:45:14.596098 24702 raft_consensus.cc:2454] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9552fb76d538450791575265a9240c9e in term 1.
I20260619 06:45:14.596103 24800 consensus_queue.cc:237] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.599066 24265 catalog_manager.cc:5495] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9552fb76d538450791575265a9240c9e (127.16.79.65). New cstate: current_term: 1 leader_uuid: "9552fb76d538450791575265a9240c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } health_report { overall_health: UNKNOWN } } }
Jun 19 06:45:14 dist-test-slave-5w2x krb5kdc[24220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851510, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.126@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM
I20260619 06:45:14.680234 24814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:14.666457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38019 (local address 127.0.0.1:46042)
0619 06:45:14.667147 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:14.667172 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:14.667336 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:14.667761 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:14.667767 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:14.668522 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:14.668722 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:14.668731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.669801 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.669810 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:14.670729 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:14.670747 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.671463 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.671469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:14.671658 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:14.672364 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:14.672386 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:14.674476 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:14.677394 (+ 2918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:14.677408 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:14.677417 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:14.677911 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:14.678294 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:14.678301 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:14.678304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:14.678429 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:14.678980 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:14.678988 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:14.679149 (+ 161us) client_negotiation.cc:769] Sending connection context
0619 06:45:14.679364 (+ 215us) client_negotiation.cc:240] Negotiation successful
0619 06:45:14.679698 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":25,"thread_start_us":333,"threads_started":1}
I20260619 06:45:14.680261 24815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:14.666489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46042 (local address 127.16.79.126:38019)
0619 06:45:14.667460 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:14.667465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:14.667481 (+ 16us) server_negotiation.cc:407] Connection header received
0619 06:45:14.667519 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:14.667522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:14.667629 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:14.667732 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:14.668881 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.669660 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.670870 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.671306 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.671441 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:14.674674 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:14.674679 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:14.674682 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:14.674709 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:14.677264 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:14.678019 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:14.678026 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:14.678031 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:14.678183 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:14.678526 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:14.678534 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:14.678540 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:14.678856 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:14.678991 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:14.679443 (+ 452us) server_negotiation.cc:299] Negotiation successful
0619 06:45:14.679830 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":182,"server-negotiator.queue_time_us":838,"thread_start_us":649,"threads_started":1}
I20260619 06:45:14.685645 24265 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:46042:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 06:45:14.686667 24265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:45:14.693540 24402 tablet_service.cc:1467] Processing CreateTablet for tablet 01f27604ceb844ab854501c467e38bbd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8e59e4ffdb144dfab448b47b86bc5e41]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:45:14.693951 24542 tablet_service.cc:1467] Processing CreateTablet for tablet 01f27604ceb844ab854501c467e38bbd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8e59e4ffdb144dfab448b47b86bc5e41]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:45:14.694164 24402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01f27604ceb844ab854501c467e38bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:14.694479 24542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01f27604ceb844ab854501c467e38bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:14.695243 24682 tablet_service.cc:1467] Processing CreateTablet for tablet 01f27604ceb844ab854501c467e38bbd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8e59e4ffdb144dfab448b47b86bc5e41]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 06:45:14.695761 24682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01f27604ceb844ab854501c467e38bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:14.701930 24778 tablet_bootstrap.cc:492] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e: Bootstrap starting.
I20260619 06:45:14.702711 24779 tablet_bootstrap.cc:492] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0: Bootstrap starting.
I20260619 06:45:14.703783 24778 tablet_bootstrap.cc:654] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:14.704383 24779 tablet_bootstrap.cc:654] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:14.712375 24777 tablet_bootstrap.cc:492] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3: Bootstrap starting.
I20260619 06:45:14.714241 24777 tablet_bootstrap.cc:654] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:14.726382 24777 tablet_bootstrap.cc:492] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3: No bootstrap required, opened a new log
I20260619 06:45:14.726629 24777 ts_tablet_manager.cc:1397] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.004s
I20260619 06:45:14.727416 24777 raft_consensus.cc:348] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.727699 24777 raft_consensus.cc:374] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:14.727828 24777 raft_consensus.cc:729] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8a35995e3cc44d58a1cd2228c740fa3, State: Initialized, Role: FOLLOWER
I20260619 06:45:14.728170 24777 consensus_queue.cc:260] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.728680 24784 tablet_replica.cc:439] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } } }
I20260619 06:45:14.729059 24784 tablet_replica.cc:442] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:45:14.730129 24777 ts_tablet_manager.cc:1428] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 06:45:14.734331 24778 tablet_bootstrap.cc:492] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e: No bootstrap required, opened a new log
I20260619 06:45:14.734624 24778 ts_tablet_manager.cc:1397] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.012s
I20260619 06:45:14.735185 24779 tablet_bootstrap.cc:492] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0: No bootstrap required, opened a new log
I20260619 06:45:14.735419 24779 ts_tablet_manager.cc:1397] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0: Time spent bootstrapping tablet: real 0.033s user 0.001s sys 0.026s
I20260619 06:45:14.735528 24778 raft_consensus.cc:348] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.735877 24778 raft_consensus.cc:374] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:14.736004 24778 raft_consensus.cc:729] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9552fb76d538450791575265a9240c9e, State: Initialized, Role: FOLLOWER
I20260619 06:45:14.736222 24779 raft_consensus.cc:348] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.736344 24778 consensus_queue.cc:260] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.736526 24779 raft_consensus.cc:374] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:14.736593 24779 raft_consensus.cc:729] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9afac029e4e456b85384f7adbdb84d0, State: Initialized, Role: FOLLOWER
I20260619 06:45:14.736867 24779 consensus_queue.cc:260] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.736881 24800 tablet_replica.cc:439] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } } }
I20260619 06:45:14.737181 24800 tablet_replica.cc:442] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:45:14.737349 24785 tablet_replica.cc:439] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } } }
I20260619 06:45:14.737682 24785 tablet_replica.cc:442] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:45:14.738539 24779 ts_tablet_manager.cc:1428] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260619 06:45:14.740015 24778 ts_tablet_manager.cc:1428] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260619 06:45:14.821830 24800 raft_consensus.cc:482] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:45:14.822057 24800 raft_consensus.cc:504] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.822724 24800 leader_election.cc:288] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8a35995e3cc44d58a1cd2228c740fa3 (127.16.79.66:36045), c9afac029e4e456b85384f7adbdb84d0 (127.16.79.67:38139)
I20260619 06:45:14.823297 24702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01f27604ceb844ab854501c467e38bbd" candidate_uuid: "9552fb76d538450791575265a9240c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9afac029e4e456b85384f7adbdb84d0" is_pre_election: true
I20260619 06:45:14.823300 24563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01f27604ceb844ab854501c467e38bbd" candidate_uuid: "9552fb76d538450791575265a9240c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" is_pre_election: true
I20260619 06:45:14.823583 24702 raft_consensus.cc:2454] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9552fb76d538450791575265a9240c9e in term 0.
I20260619 06:45:14.823601 24563 raft_consensus.cc:2454] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9552fb76d538450791575265a9240c9e in term 0.
I20260619 06:45:14.824046 24356 leader_election.cc:302] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9552fb76d538450791575265a9240c9e, c9afac029e4e456b85384f7adbdb84d0; no voters:
I20260619 06:45:14.824352 24800 raft_consensus.cc:2781] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:45:14.824456 24800 raft_consensus.cc:482] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:45:14.824584 24800 raft_consensus.cc:3037] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:14.826915 24800 raft_consensus.cc:504] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.827499 24800 leader_election.cc:288] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [CANDIDATE]: Term 1 election: Requested vote from peers a8a35995e3cc44d58a1cd2228c740fa3 (127.16.79.66:36045), c9afac029e4e456b85384f7adbdb84d0 (127.16.79.67:38139)
I20260619 06:45:14.827934 24563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01f27604ceb844ab854501c467e38bbd" candidate_uuid: "9552fb76d538450791575265a9240c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8a35995e3cc44d58a1cd2228c740fa3"
I20260619 06:45:14.827934 24702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01f27604ceb844ab854501c467e38bbd" candidate_uuid: "9552fb76d538450791575265a9240c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9afac029e4e456b85384f7adbdb84d0"
I20260619 06:45:14.828190 24563 raft_consensus.cc:3037] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:14.828188 24702 raft_consensus.cc:3037] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:14.831041 24702 raft_consensus.cc:2454] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9552fb76d538450791575265a9240c9e in term 1.
I20260619 06:45:14.831041 24563 raft_consensus.cc:2454] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9552fb76d538450791575265a9240c9e in term 1.
I20260619 06:45:14.831472 24355 leader_election.cc:302] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9552fb76d538450791575265a9240c9e, a8a35995e3cc44d58a1cd2228c740fa3; no voters:
I20260619 06:45:14.831744 24800 raft_consensus.cc:2781] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:45:14.831924 24800 raft_consensus.cc:686] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [term 1 LEADER]: Becoming Leader. State: Replica: 9552fb76d538450791575265a9240c9e, State: Running, Role: LEADER
I20260619 06:45:14.832212 24800 consensus_queue.cc:237] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:14.833755 24788 tablet_replica.cc:439] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e: TxnStatusTablet state changed. Reason: New leader 9552fb76d538450791575265a9240c9e. Latest consensus state: current_term: 1 leader_uuid: "9552fb76d538450791575265a9240c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } } }
I20260619 06:45:14.834036 24788 tablet_replica.cc:442] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:45:14.834619 24820 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 06:45:14.834818 24820 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 06:45:14.835115 24265 catalog_manager.cc:5495] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9552fb76d538450791575265a9240c9e (127.16.79.65). New cstate: current_term: 1 leader_uuid: "9552fb76d538450791575265a9240c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } health_report { overall_health: UNKNOWN } } }
I20260619 06:45:14.868216 24814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:14.863056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:35559 (local address 127.0.0.1:53094)
0619 06:45:14.863257 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:14.863281 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:14.863429 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:14.863931 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:14.863936 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:14.863947 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:45:14.864125 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:14.864173 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.865177 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.865186 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:14.866172 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:14.866182 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.866920 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.866926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:14.867123 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:14.867174 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:45:14.867606 (+ 432us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:45:14.867626 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 06:45:14.867715 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 06:45:14.867884 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 06:45:14.868250 24822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:14.863086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53094 (local address 127.16.79.65:35559)
0619 06:45:14.863657 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:14.863662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:14.863676 (+ 14us) server_negotiation.cc:407] Connection header received
0619 06:45:14.863730 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:14.863733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:14.863790 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:14.863892 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:45:14.864298 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.865043 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.866306 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.866779 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.866911 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:14.867398 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:45:14.867514 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:45:14.867605 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:14.867747 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 06:45:14.867822 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":308,"threads_started":1}
I20260619 06:45:14.872072 24563 raft_consensus.cc:1261] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Refusing update from remote peer 9552fb76d538450791575265a9240c9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:45:14.872164 24702 raft_consensus.cc:1261] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Refusing update from remote peer 9552fb76d538450791575265a9240c9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:45:14.872713 24788 consensus_queue.cc:1035] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:45:14.873028 24800 consensus_queue.cc:1035] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:45:14.881824 24784 tablet_replica.cc:439] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3: TxnStatusTablet state changed. Reason: New leader 9552fb76d538450791575265a9240c9e. Latest consensus state: current_term: 1 leader_uuid: "9552fb76d538450791575265a9240c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } } }
I20260619 06:45:14.882109 24784 tablet_replica.cc:442] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:45:14.888161 24800 tablet_replica.cc:439] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9552fb76d538450791575265a9240c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } } }
I20260619 06:45:14.888634 24800 tablet_replica.cc:442] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:45:14.896963 24785 tablet_replica.cc:439] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0: TxnStatusTablet state changed. Reason: New leader 9552fb76d538450791575265a9240c9e. Latest consensus state: current_term: 1 leader_uuid: "9552fb76d538450791575265a9240c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } } }
I20260619 06:45:14.897249 24785 tablet_replica.cc:442] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:45:14.900430 24784 tablet_replica.cc:439] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9552fb76d538450791575265a9240c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } } }
I20260619 06:45:14.900682 24784 tablet_replica.cc:442] T 01f27604ceb844ab854501c467e38bbd P a8a35995e3cc44d58a1cd2228c740fa3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:45:14.901590 24788 tablet_replica.cc:439] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9552fb76d538450791575265a9240c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } } }
I20260619 06:45:14.901873 24788 tablet_replica.cc:442] T 01f27604ceb844ab854501c467e38bbd P 9552fb76d538450791575265a9240c9e: This TxnStatusTablet replica's current role is: LEADER
I20260619 06:45:14.903213 24824 mvcc.cc:205] Tried to move back new op lower bound from 7298463804910317568 to 7298463804755083264. Current Snapshot: MvccSnapshot[applied={T|T < 7298463804910317568}]
I20260619 06:45:14.905061 24785 tablet_replica.cc:439] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9552fb76d538450791575265a9240c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } } }
I20260619 06:45:14.905298 24785 tablet_replica.cc:442] T 01f27604ceb844ab854501c467e38bbd P c9afac029e4e456b85384f7adbdb84d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 06:45:14.915823 24822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:14.909870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:52153 (local address 127.16.79.65:35559)
0619 06:45:14.910055 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:14.910064 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:14.910659 (+ 595us) server_negotiation.cc:407] Connection header received
0619 06:45:14.910954 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:14.910958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:14.911023 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:14.911101 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:45:14.911651 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.912559 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.914127 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.914540 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.914678 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:14.915096 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:45:14.915207 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:45:14.915303 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:14.915417 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 06:45:14.915490 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:45:14.915920 24833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:14.909829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:35559 (local address 127.16.79.65:52153)
0619 06:45:14.910541 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:14.910561 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:14.910720 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:14.911145 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:14.911151 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:14.911171 (+ 20us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 06:45:14.911500 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:14.911516 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.912702 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.912717 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:14.913894 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:14.913905 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:14.914723 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:14.914730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:14.914874 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:14.914943 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 06:45:14.915301 (+ 358us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 06:45:14.915310 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 06:45:14.915402 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 06:45:14.915523 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":361,"threads_started":1}
I20260619 06:45:14.924713 24702 raft_consensus.cc:1261] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Refusing update from remote peer 9552fb76d538450791575265a9240c9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:45:14.925414 24788 consensus_queue.cc:1035] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:45:14.934387 24563 raft_consensus.cc:1261] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Refusing update from remote peer 9552fb76d538450791575265a9240c9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:45:14.935915 24800 consensus_queue.cc:1035] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:45:14.941239 24825 mvcc.cc:205] Tried to move back new op lower bound from 7298463805126246400 to 7298463803788279808. Current Snapshot: MvccSnapshot[applied={T|T < 7298463805126246400}]
I20260619 06:45:14.992959 24265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:45:14.993206 24265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:45:14.996448 24265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b: Sending DeleteTablet for 3 replicas of tablet e0b10272864546eda9657257374e3048
I20260619 06:45:14.997406 24682 tablet_service.cc:1514] Processing DeleteTablet for tablet e0b10272864546eda9657257374e3048 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:14 UTC) from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:37886
I20260619 06:45:14.997722 24403 tablet_service.cc:1514] Processing DeleteTablet for tablet e0b10272864546eda9657257374e3048 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:14 UTC) from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:58000
I20260619 06:45:14.999403 24839 tablet_replica.cc:330] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e: stopping tablet replica
I20260619 06:45:14.999679 24839 raft_consensus.cc:2229] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:45:15.000092 24839 raft_consensus.cc:2258] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:15.001082 24839 ts_tablet_manager.cc:1905] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:15.001044 24542 tablet_service.cc:1514] Processing DeleteTablet for tablet e0b10272864546eda9657257374e3048 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:14 UTC) from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:40424
I20260619 06:45:15.009809 24839 ts_tablet_manager.cc:1918] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:45:15.009977 24839 log.cc:1196] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/e0b10272864546eda9657257374e3048
I20260619 06:45:15.010426 24838 tablet_replica.cc:330] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0: stopping tablet replica
I20260619 06:45:15.010673 24839 ts_tablet_manager.cc:1939] T e0b10272864546eda9657257374e3048 P 9552fb76d538450791575265a9240c9e: Deleting consensus metadata
I20260619 06:45:15.010828 24838 raft_consensus.cc:2229] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:45:15.011193 24838 raft_consensus.cc:2258] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:15.012058 24250 catalog_manager.cc:4841] TS 9552fb76d538450791575265a9240c9e (127.16.79.65:35559): tablet e0b10272864546eda9657257374e3048 (table test-table [id=ff8ba6d402c24c07bcb96825b9e09a73]) successfully deleted
I20260619 06:45:15.012724 24838 ts_tablet_manager.cc:1905] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:15.019215 24840 tablet_replica.cc:330] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3: stopping tablet replica
I20260619 06:45:15.019510 24840 raft_consensus.cc:2229] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:45:15.019676 24838 ts_tablet_manager.cc:1918] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:45:15.019805 24838 log.cc:1196] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/e0b10272864546eda9657257374e3048
I20260619 06:45:15.019829 24840 raft_consensus.cc:2258] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:15.020334 24838 ts_tablet_manager.cc:1939] T e0b10272864546eda9657257374e3048 P c9afac029e4e456b85384f7adbdb84d0: Deleting consensus metadata
I20260619 06:45:15.021381 24252 catalog_manager.cc:4841] TS c9afac029e4e456b85384f7adbdb84d0 (127.16.79.67:38139): tablet e0b10272864546eda9657257374e3048 (table test-table [id=ff8ba6d402c24c07bcb96825b9e09a73]) successfully deleted
I20260619 06:45:15.024920 24840 ts_tablet_manager.cc:1905] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:15.031905 24840 ts_tablet_manager.cc:1918] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:45:15.032049 24840 log.cc:1196] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/e0b10272864546eda9657257374e3048
I20260619 06:45:15.032495 24840 ts_tablet_manager.cc:1939] T e0b10272864546eda9657257374e3048 P a8a35995e3cc44d58a1cd2228c740fa3: Deleting consensus metadata
I20260619 06:45:15.033438 24251 catalog_manager.cc:4841] TS a8a35995e3cc44d58a1cd2228c740fa3 (127.16.79.66:36045): tablet e0b10272864546eda9657257374e3048 (table test-table [id=ff8ba6d402c24c07bcb96825b9e09a73]) successfully deleted
Jun 19 06:45:15 dist-test-slave-5w2x krb5kdc[24220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 06:45:15 dist-test-slave-5w2x krb5kdc[24220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:15 dist-test-slave-5w2x krb5kdc[24220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 06:45:15.067282 24815 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:15.055710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46054 (local address 127.16.79.126:38019)
0619 06:45:15.055958 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:15.055965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:15.056313 (+ 348us) server_negotiation.cc:407] Connection header received
0619 06:45:15.056482 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:15.056488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:15.056574 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:15.056704 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:15.057994 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.059134 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.060470 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.061163 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.061301 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:15.066956 (+ 5655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46054: BlockingRecv error: recv got EOF from 127.0.0.1:46054 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 06:45:15 dist-test-slave-5w2x krb5kdc[24220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM
I20260619 06:45:15.087714 24815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:15.070172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.16.79.126:38019)
0619 06:45:15.070356 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:15.070363 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:15.072698 (+ 2335us) server_negotiation.cc:407] Connection header received
0619 06:45:15.072900 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:15.072904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:15.072971 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:15.073057 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:15.074599 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.075292 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.077091 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.077566 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.077659 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:15.082484 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:15.082490 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:15.082493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:15.082528 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:15.085124 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:15.085798 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:15.085803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:15.085806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:15.085934 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:15.086325 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:15.086330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:15.086334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:15.086595 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:15.086860 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:15.087152 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 06:45:15.087374 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 06:45:15.091423 24265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:45:15.092181 24265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:45:15.098888 24403 tablet_service.cc:1467] Processing CreateTablet for tablet e4c65b6e88ca44f68d6f3dd170ec5a72 (DEFAULT_TABLE table=test-table [id=5cfc59f31f254bb88ae58a29a2fc3d9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:15.099272 24682 tablet_service.cc:1467] Processing CreateTablet for tablet e4c65b6e88ca44f68d6f3dd170ec5a72 (DEFAULT_TABLE table=test-table [id=5cfc59f31f254bb88ae58a29a2fc3d9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:15.099264 24542 tablet_service.cc:1467] Processing CreateTablet for tablet e4c65b6e88ca44f68d6f3dd170ec5a72 (DEFAULT_TABLE table=test-table [id=5cfc59f31f254bb88ae58a29a2fc3d9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:15.099391 24403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4c65b6e88ca44f68d6f3dd170ec5a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:15.099660 24682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4c65b6e88ca44f68d6f3dd170ec5a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:15.099680 24542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4c65b6e88ca44f68d6f3dd170ec5a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:15.105953 24778 tablet_bootstrap.cc:492] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e: Bootstrap starting.
I20260619 06:45:15.106282 24777 tablet_bootstrap.cc:492] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3: Bootstrap starting.
I20260619 06:45:15.106283 24779 tablet_bootstrap.cc:492] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0: Bootstrap starting.
I20260619 06:45:15.107630 24778 tablet_bootstrap.cc:654] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:15.107910 24777 tablet_bootstrap.cc:654] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:15.108033 24779 tablet_bootstrap.cc:654] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:15.119199 24779 tablet_bootstrap.cc:492] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0: No bootstrap required, opened a new log
I20260619 06:45:15.119453 24779 ts_tablet_manager.cc:1397] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20260619 06:45:15.119695 24778 tablet_bootstrap.cc:492] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e: No bootstrap required, opened a new log
I20260619 06:45:15.119865 24778 ts_tablet_manager.cc:1397] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.002s
I20260619 06:45:15.120662 24779 raft_consensus.cc:348] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.120716 24778 raft_consensus.cc:348] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.121004 24778 raft_consensus.cc:374] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:15.121106 24778 raft_consensus.cc:729] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9552fb76d538450791575265a9240c9e, State: Initialized, Role: FOLLOWER
I20260619 06:45:15.121117 24779 raft_consensus.cc:374] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:15.121356 24779 raft_consensus.cc:729] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9afac029e4e456b85384f7adbdb84d0, State: Initialized, Role: FOLLOWER
I20260619 06:45:15.121395 24778 consensus_queue.cc:260] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.122113 24777 tablet_bootstrap.cc:492] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3: No bootstrap required, opened a new log
I20260619 06:45:15.122105 24778 ts_tablet_manager.cc:1428] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 06:45:15.122287 24777 ts_tablet_manager.cc:1397] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.004s
I20260619 06:45:15.122694 24779 consensus_queue.cc:260] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.123107 24777 raft_consensus.cc:348] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.123363 24777 raft_consensus.cc:374] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:15.123452 24777 raft_consensus.cc:729] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8a35995e3cc44d58a1cd2228c740fa3, State: Initialized, Role: FOLLOWER
I20260619 06:45:15.123663 24777 consensus_queue.cc:260] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.124655 24777 ts_tablet_manager.cc:1428] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260619 06:45:15.125124 24779 ts_tablet_manager.cc:1428] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260619 06:45:15.279441 24785 raft_consensus.cc:482] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:45:15.279667 24785 raft_consensus.cc:504] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.281224 24785 leader_election.cc:288] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9552fb76d538450791575265a9240c9e (127.16.79.65:35559), a8a35995e3cc44d58a1cd2228c740fa3 (127.16.79.66:36045)
I20260619 06:45:15.287756 24822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:15.281569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:51651 (local address 127.16.79.65:35559)
0619 06:45:15.281766 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:15.281773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:15.282443 (+ 670us) server_negotiation.cc:407] Connection header received
0619 06:45:15.282757 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:15.282763 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:15.282835 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:15.282945 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:15.283404 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.284320 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.286020 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.286781 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.286965 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:15.287045 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:15.287302 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 06:45:15.287442 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:45:15.287843 24857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:15.281490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:35559 (local address 127.16.79.67:51651)
0619 06:45:15.282329 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:15.282353 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:15.282574 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:15.282968 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:15.282973 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:15.283021 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:15.283260 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:15.283268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.284456 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.284465 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:15.285858 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:15.285870 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.286971 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.286977 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:15.287143 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:15.287166 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 06:45:15.287276 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 06:45:15.287414 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":1,"spinlock_wait_cycles":1408,"thread_start_us":1061,"threads_started":1}
I20260619 06:45:15.288166 24858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:15.281925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:34415 (local address 127.16.79.66:36045)
0619 06:45:15.282556 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:15.282562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:15.282947 (+ 385us) server_negotiation.cc:407] Connection header received
0619 06:45:15.283150 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:15.283154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:15.283212 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:15.283316 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:15.283850 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.284650 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.286348 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.287148 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.287364 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:15.287436 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:15.287669 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 06:45:15.287804 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":389,"threads_started":1}
I20260619 06:45:15.288668 24856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:15.281680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:36045 (local address 127.16.79.67:34415)
0619 06:45:15.282837 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:15.282854 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:15.283017 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:15.283331 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:15.283335 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:15.283359 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:15.283663 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:15.283671 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.284808 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.284815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:15.286189 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:15.286200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.287364 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.287370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:15.287522 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:15.287547 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 06:45:15.287647 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 06:45:15.287776 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1053,"spinlock_wait_cycles":3840,"thread_start_us":356,"threads_started":1}
I20260619 06:45:15.288769 24427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4c65b6e88ca44f68d6f3dd170ec5a72" candidate_uuid: "c9afac029e4e456b85384f7adbdb84d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9552fb76d538450791575265a9240c9e" is_pre_election: true
I20260619 06:45:15.289213 24427 raft_consensus.cc:2454] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9afac029e4e456b85384f7adbdb84d0 in term 0.
I20260619 06:45:15.289486 24562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4c65b6e88ca44f68d6f3dd170ec5a72" candidate_uuid: "c9afac029e4e456b85384f7adbdb84d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" is_pre_election: true
I20260619 06:45:15.289799 24562 raft_consensus.cc:2454] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9afac029e4e456b85384f7adbdb84d0 in term 0.
I20260619 06:45:15.289875 24634 leader_election.cc:302] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9552fb76d538450791575265a9240c9e, c9afac029e4e456b85384f7adbdb84d0; no voters:
I20260619 06:45:15.290337 24785 raft_consensus.cc:2781] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:45:15.290475 24785 raft_consensus.cc:482] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:45:15.290576 24785 raft_consensus.cc:3037] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:15.292762 24785 raft_consensus.cc:504] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.293347 24785 leader_election.cc:288] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [CANDIDATE]: Term 1 election: Requested vote from peers 9552fb76d538450791575265a9240c9e (127.16.79.65:35559), a8a35995e3cc44d58a1cd2228c740fa3 (127.16.79.66:36045)
I20260619 06:45:15.293773 24427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4c65b6e88ca44f68d6f3dd170ec5a72" candidate_uuid: "c9afac029e4e456b85384f7adbdb84d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9552fb76d538450791575265a9240c9e"
I20260619 06:45:15.293824 24562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4c65b6e88ca44f68d6f3dd170ec5a72" candidate_uuid: "c9afac029e4e456b85384f7adbdb84d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8a35995e3cc44d58a1cd2228c740fa3"
I20260619 06:45:15.293999 24427 raft_consensus.cc:3037] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:15.294041 24562 raft_consensus.cc:3037] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:15.296039 24562 raft_consensus.cc:2454] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9afac029e4e456b85384f7adbdb84d0 in term 1.
I20260619 06:45:15.296096 24427 raft_consensus.cc:2454] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9afac029e4e456b85384f7adbdb84d0 in term 1.
I20260619 06:45:15.296519 24635 leader_election.cc:302] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a8a35995e3cc44d58a1cd2228c740fa3, c9afac029e4e456b85384f7adbdb84d0; no voters:
I20260619 06:45:15.296828 24785 raft_consensus.cc:2781] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:45:15.297266 24785 raft_consensus.cc:686] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [term 1 LEADER]: Becoming Leader. State: Replica: c9afac029e4e456b85384f7adbdb84d0, State: Running, Role: LEADER
I20260619 06:45:15.297631 24785 consensus_queue.cc:237] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.301385 24264 catalog_manager.cc:5495] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to c9afac029e4e456b85384f7adbdb84d0 (127.16.79.67). New cstate: current_term: 1 leader_uuid: "c9afac029e4e456b85384f7adbdb84d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } health_report { overall_health: HEALTHY } } }
I20260619 06:45:15.317569 24862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:15.311777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46938 (local address 127.16.79.67:38139)
0619 06:45:15.312353 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:15.312358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:15.312377 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:45:15.312452 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:15.312456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:15.312528 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:15.312690 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:45:15.313114 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.313876 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.315217 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.315683 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.315814 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:15.316338 (+ 524us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:45:15.316927 (+ 589us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:45:15.317083 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:15.317161 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 06:45:15.317254 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":280,"threads_started":1}
I20260619 06:45:15.320906 24562 raft_consensus.cc:1261] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Refusing update from remote peer c9afac029e4e456b85384f7adbdb84d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:45:15.321182 24427 raft_consensus.cc:1261] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e [term 1 FOLLOWER]: Refusing update from remote peer c9afac029e4e456b85384f7adbdb84d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:45:15.321591 24859 consensus_queue.cc:1035] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:45:15.322023 24785 consensus_queue.cc:1035] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:45:15.337168 24264 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:45:15.337437 24264 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:45:15.341568 24264 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b: Sending DeleteTablet for 3 replicas of tablet e4c65b6e88ca44f68d6f3dd170ec5a72
I20260619 06:45:15.343032 24682 tablet_service.cc:1514] Processing DeleteTablet for tablet e4c65b6e88ca44f68d6f3dd170ec5a72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:15 UTC) from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:37886
I20260619 06:45:15.343055 24542 tablet_service.cc:1514] Processing DeleteTablet for tablet e4c65b6e88ca44f68d6f3dd170ec5a72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:15 UTC) from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:40424
I20260619 06:45:15.343366 24840 tablet_replica.cc:330] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3: stopping tablet replica
I20260619 06:45:15.343423 24403 tablet_service.cc:1514] Processing DeleteTablet for tablet e4c65b6e88ca44f68d6f3dd170ec5a72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:15 UTC) from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:58000
I20260619 06:45:15.343568 24840 raft_consensus.cc:2229] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:45:15.343684 24402 tablet_service.cc:1514] Processing DeleteTablet for tablet e4c65b6e88ca44f68d6f3dd170ec5a72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:15 UTC) from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:58000
I20260619 06:45:15.343791 24840 raft_consensus.cc:2258] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:15.343822 24265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:45:15.344481 24265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:45:15.344527 24838 tablet_replica.cc:330] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0: stopping tablet replica
I20260619 06:45:15.344734 24838 raft_consensus.cc:2229] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:45:15.345291 24838 raft_consensus.cc:2258] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:15.345367 24840 ts_tablet_manager.cc:1905] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:15.345554 24839 tablet_replica.cc:330] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e: stopping tablet replica
I20260619 06:45:15.345737 24839 raft_consensus.cc:2229] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:45:15.345963 24839 raft_consensus.cc:2258] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:15.349073 24838 ts_tablet_manager.cc:1905] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:15.353027 24839 ts_tablet_manager.cc:1905] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:15.355346 24840 ts_tablet_manager.cc:1918] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:45:15.355517 24840 log.cc:1196] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/e4c65b6e88ca44f68d6f3dd170ec5a72
I20260619 06:45:15.355638 24838 ts_tablet_manager.cc:1918] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:45:15.355796 24838 log.cc:1196] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/e4c65b6e88ca44f68d6f3dd170ec5a72
I20260619 06:45:15.355928 24840 ts_tablet_manager.cc:1939] T e4c65b6e88ca44f68d6f3dd170ec5a72 P a8a35995e3cc44d58a1cd2228c740fa3: Deleting consensus metadata
I20260619 06:45:15.356278 24838 ts_tablet_manager.cc:1939] T e4c65b6e88ca44f68d6f3dd170ec5a72 P c9afac029e4e456b85384f7adbdb84d0: Deleting consensus metadata
I20260619 06:45:15.357414 24252 catalog_manager.cc:4841] TS c9afac029e4e456b85384f7adbdb84d0 (127.16.79.67:38139): tablet e4c65b6e88ca44f68d6f3dd170ec5a72 (table test-table [id=5cfc59f31f254bb88ae58a29a2fc3d9e]) successfully deleted
I20260619 06:45:15.357822 24682 tablet_service.cc:1467] Processing CreateTablet for tablet 07bc4a680bd747778efa7bbafbcba2dd (DEFAULT_TABLE table=test-table [id=f7b96c20c61d4879a46044175e6c5303]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:15.357821 24402 tablet_service.cc:1467] Processing CreateTablet for tablet 07bc4a680bd747778efa7bbafbcba2dd (DEFAULT_TABLE table=test-table [id=f7b96c20c61d4879a46044175e6c5303]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:15.357908 24542 tablet_service.cc:1467] Processing CreateTablet for tablet 07bc4a680bd747778efa7bbafbcba2dd (DEFAULT_TABLE table=test-table [id=f7b96c20c61d4879a46044175e6c5303]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:15.358193 24542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07bc4a680bd747778efa7bbafbcba2dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:15.358215 24682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07bc4a680bd747778efa7bbafbcba2dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:15.358227 24402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07bc4a680bd747778efa7bbafbcba2dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:15.363267 24251 catalog_manager.cc:4841] TS a8a35995e3cc44d58a1cd2228c740fa3 (127.16.79.66:36045): tablet e4c65b6e88ca44f68d6f3dd170ec5a72 (table test-table [id=5cfc59f31f254bb88ae58a29a2fc3d9e]) successfully deleted
I20260619 06:45:15.363845 24839 ts_tablet_manager.cc:1918] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:45:15.363983 24777 tablet_bootstrap.cc:492] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3: Bootstrap starting.
I20260619 06:45:15.364024 24839 log.cc:1196] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/e4c65b6e88ca44f68d6f3dd170ec5a72
I20260619 06:45:15.364507 24839 ts_tablet_manager.cc:1939] T e4c65b6e88ca44f68d6f3dd170ec5a72 P 9552fb76d538450791575265a9240c9e: Deleting consensus metadata
I20260619 06:45:15.364560 24778 tablet_bootstrap.cc:492] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e: Bootstrap starting.
I20260619 06:45:15.365669 24777 tablet_bootstrap.cc:654] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:15.366147 24778 tablet_bootstrap.cc:654] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:15.367246 24250 catalog_manager.cc:4841] TS 9552fb76d538450791575265a9240c9e (127.16.79.65:35559): tablet e4c65b6e88ca44f68d6f3dd170ec5a72 (table test-table [id=5cfc59f31f254bb88ae58a29a2fc3d9e]) successfully deleted
W20260619 06:45:15.367659 24250 catalog_manager.cc:4808] TS 9552fb76d538450791575265a9240c9e (127.16.79.65:35559): delete failed for tablet e4c65b6e88ca44f68d6f3dd170ec5a72 because the tablet was not found. No further retry: Not found: Tablet not found: e4c65b6e88ca44f68d6f3dd170ec5a72
I20260619 06:45:15.369122 24779 tablet_bootstrap.cc:492] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0: Bootstrap starting.
I20260619 06:45:15.370265 24777 tablet_bootstrap.cc:492] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3: No bootstrap required, opened a new log
I20260619 06:45:15.370498 24777 ts_tablet_manager.cc:1397] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260619 06:45:15.370728 24778 tablet_bootstrap.cc:492] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e: No bootstrap required, opened a new log
I20260619 06:45:15.370734 24779 tablet_bootstrap.cc:654] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:15.370903 24778 ts_tablet_manager.cc:1397] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260619 06:45:15.371441 24777 raft_consensus.cc:348] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.371804 24777 raft_consensus.cc:374] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:15.371670 24778 raft_consensus.cc:348] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.371980 24778 raft_consensus.cc:374] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:15.371986 24777 raft_consensus.cc:729] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8a35995e3cc44d58a1cd2228c740fa3, State: Initialized, Role: FOLLOWER
I20260619 06:45:15.372077 24778 raft_consensus.cc:729] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9552fb76d538450791575265a9240c9e, State: Initialized, Role: FOLLOWER
I20260619 06:45:15.372342 24777 consensus_queue.cc:260] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.372424 24778 consensus_queue.cc:260] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.373220 24778 ts_tablet_manager.cc:1428] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260619 06:45:15.375178 24777 ts_tablet_manager.cc:1428] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260619 06:45:15.379068 24779 tablet_bootstrap.cc:492] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0: No bootstrap required, opened a new log
I20260619 06:45:15.379325 24779 ts_tablet_manager.cc:1397] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260619 06:45:15.380081 24779 raft_consensus.cc:348] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.380385 24779 raft_consensus.cc:374] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:15.380486 24779 raft_consensus.cc:729] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9afac029e4e456b85384f7adbdb84d0, State: Initialized, Role: FOLLOWER
I20260619 06:45:15.380774 24779 consensus_queue.cc:260] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.381268 24779 ts_tablet_manager.cc:1428] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 06:45:15.411442 24800 raft_consensus.cc:482] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:45:15.411614 24800 raft_consensus.cc:504] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.412576 24800 leader_election.cc:288] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8a35995e3cc44d58a1cd2228c740fa3 (127.16.79.66:36045), c9afac029e4e456b85384f7adbdb84d0 (127.16.79.67:38139)
I20260619 06:45:15.413007 24562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07bc4a680bd747778efa7bbafbcba2dd" candidate_uuid: "9552fb76d538450791575265a9240c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" is_pre_election: true
I20260619 06:45:15.413272 24562 raft_consensus.cc:2454] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9552fb76d538450791575265a9240c9e in term 0.
I20260619 06:45:15.413295 24702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07bc4a680bd747778efa7bbafbcba2dd" candidate_uuid: "9552fb76d538450791575265a9240c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9afac029e4e456b85384f7adbdb84d0" is_pre_election: true
I20260619 06:45:15.413527 24702 raft_consensus.cc:2454] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9552fb76d538450791575265a9240c9e in term 0.
I20260619 06:45:15.413681 24355 leader_election.cc:302] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9552fb76d538450791575265a9240c9e, a8a35995e3cc44d58a1cd2228c740fa3; no voters:
I20260619 06:45:15.414000 24800 raft_consensus.cc:2781] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:45:15.414122 24800 raft_consensus.cc:482] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:45:15.414225 24800 raft_consensus.cc:3037] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:15.416513 24800 raft_consensus.cc:504] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.417083 24800 leader_election.cc:288] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [CANDIDATE]: Term 1 election: Requested vote from peers a8a35995e3cc44d58a1cd2228c740fa3 (127.16.79.66:36045), c9afac029e4e456b85384f7adbdb84d0 (127.16.79.67:38139)
I20260619 06:45:15.417508 24562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07bc4a680bd747778efa7bbafbcba2dd" candidate_uuid: "9552fb76d538450791575265a9240c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8a35995e3cc44d58a1cd2228c740fa3"
I20260619 06:45:15.417548 24702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07bc4a680bd747778efa7bbafbcba2dd" candidate_uuid: "9552fb76d538450791575265a9240c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9afac029e4e456b85384f7adbdb84d0"
I20260619 06:45:15.417706 24562 raft_consensus.cc:3037] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:15.417763 24702 raft_consensus.cc:3037] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:15.419611 24562 raft_consensus.cc:2454] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9552fb76d538450791575265a9240c9e in term 1.
I20260619 06:45:15.420027 24355 leader_election.cc:302] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9552fb76d538450791575265a9240c9e, a8a35995e3cc44d58a1cd2228c740fa3; no voters:
I20260619 06:45:15.420301 24800 raft_consensus.cc:2781] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:45:15.420460 24800 raft_consensus.cc:686] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [term 1 LEADER]: Becoming Leader. State: Replica: 9552fb76d538450791575265a9240c9e, State: Running, Role: LEADER
I20260619 06:45:15.420821 24702 raft_consensus.cc:2454] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9552fb76d538450791575265a9240c9e in term 1.
I20260619 06:45:15.420734 24800 consensus_queue.cc:237] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } }
I20260619 06:45:15.423388 24265 catalog_manager.cc:5495] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9552fb76d538450791575265a9240c9e (127.16.79.65). New cstate: current_term: 1 leader_uuid: "9552fb76d538450791575265a9240c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9552fb76d538450791575265a9240c9e" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 } health_report { overall_health: UNKNOWN } } }
I20260619 06:45:15.459501 24822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:15.454119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53106 (local address 127.16.79.65:35559)
0619 06:45:15.454285 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:15.454292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:15.454562 (+ 270us) server_negotiation.cc:407] Connection header received
0619 06:45:15.454728 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:15.454732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:15.454793 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:15.454863 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:45:15.455302 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.455976 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.457884 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.458278 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.458370 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:15.458846 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:45:15.458938 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:45:15.459000 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:15.459152 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 06:45:15.459210 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 06:45:15.468283 24562 raft_consensus.cc:1261] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Refusing update from remote peer 9552fb76d538450791575265a9240c9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:45:15.468564 24702 raft_consensus.cc:1261] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Refusing update from remote peer 9552fb76d538450791575265a9240c9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:45:15.468854 24834 consensus_queue.cc:1035] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8a35995e3cc44d58a1cd2228c740fa3" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:45:15.469177 24788 consensus_queue.cc:1035] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9afac029e4e456b85384f7adbdb84d0" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:45:15.524514 24265 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:45:15.524796 24265 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:45:15.527721 24265 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1faa3e74ca1f4504a9b45a6bc193c84b: Sending DeleteTablet for 3 replicas of tablet 07bc4a680bd747778efa7bbafbcba2dd
I20260619 06:45:15.528848 24682 tablet_service.cc:1514] Processing DeleteTablet for tablet 07bc4a680bd747778efa7bbafbcba2dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:15 UTC) from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:37886
I20260619 06:45:15.528879 24542 tablet_service.cc:1514] Processing DeleteTablet for tablet 07bc4a680bd747778efa7bbafbcba2dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:15 UTC) from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:40424
I20260619 06:45:15.529172 24838 tablet_replica.cc:330] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0: stopping tablet replica
I20260619 06:45:15.529350 24838 raft_consensus.cc:2229] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:45:15.529381 24840 tablet_replica.cc:330] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3: stopping tablet replica
I20260619 06:45:15.529546 24838 raft_consensus.cc:2258] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:15.529590 24840 raft_consensus.cc:2229] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:45:15.529809 24840 raft_consensus.cc:2258] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:15.529892 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 24337
I20260619 06:45:15.530707 24403 tablet_service.cc:1514] Processing DeleteTablet for tablet 07bc4a680bd747778efa7bbafbcba2dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:15 UTC) from {username='oryx', principal='oryx/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:58000
I20260619 06:45:15.530982 24839 tablet_replica.cc:330] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e: stopping tablet replica
I20260619 06:45:15.530982 24838 ts_tablet_manager.cc:1905] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:15.531168 24839 raft_consensus.cc:2229] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:45:15.531514 24839 raft_consensus.cc:2258] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:15.532797 24840 ts_tablet_manager.cc:1905] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:15.533370 24839 ts_tablet_manager.cc:1905] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:15.538314 24838 ts_tablet_manager.cc:1918] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:45:15.538457 24838 log.cc:1196] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/07bc4a680bd747778efa7bbafbcba2dd
I20260619 06:45:15.538837 24838 ts_tablet_manager.cc:1939] T 07bc4a680bd747778efa7bbafbcba2dd P c9afac029e4e456b85384f7adbdb84d0: Deleting consensus metadata
I20260619 06:45:15.539629 24840 ts_tablet_manager.cc:1918] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:45:15.539783 24840 log.cc:1196] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/07bc4a680bd747778efa7bbafbcba2dd
I20260619 06:45:15.539911 24252 catalog_manager.cc:4841] TS c9afac029e4e456b85384f7adbdb84d0 (127.16.79.67:38139): tablet 07bc4a680bd747778efa7bbafbcba2dd (table test-table [id=f7b96c20c61d4879a46044175e6c5303]) successfully deleted
I20260619 06:45:15.540243 24840 ts_tablet_manager.cc:1939] T 07bc4a680bd747778efa7bbafbcba2dd P a8a35995e3cc44d58a1cd2228c740fa3: Deleting consensus metadata
I20260619 06:45:15.541330 24251 catalog_manager.cc:4841] TS a8a35995e3cc44d58a1cd2228c740fa3 (127.16.79.66:36045): tablet 07bc4a680bd747778efa7bbafbcba2dd (table test-table [id=f7b96c20c61d4879a46044175e6c5303]) successfully deleted
I20260619 06:45:15.542477 24839 ts_tablet_manager.cc:1918] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 06:45:15.542618 24839 log.cc:1196] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/07bc4a680bd747778efa7bbafbcba2dd
I20260619 06:45:15.543087 24839 ts_tablet_manager.cc:1939] T 07bc4a680bd747778efa7bbafbcba2dd P 9552fb76d538450791575265a9240c9e: Deleting consensus metadata
I20260619 06:45:15.544200 24250 catalog_manager.cc:4841] TS 9552fb76d538450791575265a9240c9e (127.16.79.65:35559): tablet 07bc4a680bd747778efa7bbafbcba2dd (table test-table [id=f7b96c20c61d4879a46044175e6c5303]) successfully deleted
Jun 19 06:45:15 dist-test-slave-5w2x krb5kdc[24220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.79.65@KRBTEST.COM
I20260619 06:45:15.556879 24822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:15.536380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53112 (local address 127.16.79.65:35559)
0619 06:45:15.536555 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:15.536561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:15.541591 (+ 5030us) server_negotiation.cc:407] Connection header received
0619 06:45:15.541851 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:15.541857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:15.541957 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:15.542062 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:15.543738 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.544881 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.546067 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.546688 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.546809 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:15.551147 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:15.551161 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:15.551177 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:15.551235 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:15.554167 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:15.554960 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:15.554966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:15.554968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:15.555102 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:15.555466 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:15.555469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:15.555471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:15.555823 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:15.556031 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:15.556387 (+ 356us) server_negotiation.cc:299] Negotiation successful
0619 06:45:15.556579 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:45:15.557574 24468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:15.714582 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 24337
I20260619 06:45:15.749996 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 24477
Jun 19 06:45:15 dist-test-slave-5w2x krb5kdc[24220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.79.66@KRBTEST.COM
I20260619 06:45:15.766717 24858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:15.750894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49876 (local address 127.16.79.66:36045)
0619 06:45:15.751140 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:15.751146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:15.751185 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:45:15.751252 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:15.751257 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:15.751346 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:15.751459 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:15.752648 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.753737 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.754956 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.755613 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.755743 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:15.759888 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:15.759907 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:15.759927 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:15.759972 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:15.763758 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:15.764592 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:15.764603 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:15.764607 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:15.764773 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:15.765134 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:15.765139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:15.765142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:15.765585 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:15.765894 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:15.766090 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 06:45:15.766362 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:45:15.767572 24608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:15.863399 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 24477
I20260619 06:45:15.882351 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 24617
Jun 19 06:45:15 dist-test-slave-5w2x krb5kdc[24220](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.16.79.67@KRBTEST.COM
I20260619 06:45:15.896286 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:15.883009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46948 (local address 127.16.79.67:38139)
0619 06:45:15.883407 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:15.883412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:15.883428 (+ 16us) server_negotiation.cc:407] Connection header received
0619 06:45:15.883480 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:15.883483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:15.883540 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:15.883629 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:15.884899 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.885582 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.886762 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:15.887140 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:15.887232 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:15.891321 (+ 4089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:15.891336 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:15.891351 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:15.891387 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:15.893944 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:15.894575 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:15.894581 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:15.894583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:15.894705 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:15.895000 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:15.895003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:15.895005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:15.895346 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:15.895566 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:15.895805 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 06:45:15.895978 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":225,"threads_started":1}
I20260619 06:45:15.896932 24747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:16.020344 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 24617
I20260619 06:45:16.042055 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 24233
I20260619 06:45:16.043196 24305 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:16.156060 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 24233
2026-06-19T06:45:16Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6942 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:45:16 dist-test-slave-5w2x krb5kdc[24886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:45:16 dist-test-slave-5w2x krb5kdc[24886](info): set up 2 sockets
Jun 19 06:45:16 dist-test-slave-5w2x krb5kdc[24886](info): commencing operation
krb5kdc: starting...
W20260619 06:45:17.253168 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.002s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:45:17 dist-test-slave-5w2x krb5kdc[24886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:45:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:45:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:45:17.469434 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:39803
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:33417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:39803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:17.794972 24899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:17.795281 24899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:17.795372 24899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:17.802634 24899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:45:17.802750 24899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:17.802798 24899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:17.802860 24899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:45:17.802906 24899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:45:17.812886 24899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:33417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:39803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:39803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:17.814105 24899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:17.815330 24899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:17.827419 24904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:17.827464 24905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:17.827492 24907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:17.828227 24899 server_base.cc:1034] running on GCE node
I20260619 06:45:17.829107 24899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:17.830559 24899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:17.831791 24899 hybrid_clock.cc:648] HybridClock initialized: now 1781851517831749 us; error 52 us; skew 500 ppm
I20260619 06:45:17.832187 24899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T06:45:17Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1739 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:45:17 dist-test-slave-5w2x krb5kdc[24913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:45:17 dist-test-slave-5w2x krb5kdc[24913](info): set up 2 sockets
Jun 19 06:45:17 dist-test-slave-5w2x krb5kdc[24913](info): commencing operation
krb5kdc: starting...
W20260619 06:45:18.997179 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:45:19 dist-test-slave-5w2x krb5kdc[24913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:45:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:45:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:45:19.211953 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:36357
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:36357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:19.542981 24926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:19.543278 24926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:19.543437 24926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:19.550827 24926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:45:19.550947 24926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:19.551060 24926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:19.551160 24926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:45:19.551244 24926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:45:19.561779 24926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:36357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:36357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:19.563227 24926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:19.564632 24926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:19.576284 24934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:19.576346 24932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:19.576418 24931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:19.578147 24926 server_base.cc:1034] running on GCE node
I20260619 06:45:19.578933 24926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:19.580420 24926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:19.581624 24926 hybrid_clock.cc:648] HybridClock initialized: now 1781851519581581 us; error 32 us; skew 500 ppm
I20260619 06:45:19.581933 24926 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:19 dist-test-slave-5w2x krb5kdc[24913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851519, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:19.588990 24926 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:45:19.590574 24926 webserver.cc:460] Webserver started at http://127.16.79.126:33929/ using document root <none> and password file <none>
I20260619 06:45:19.591289 24926 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:19.591434 24926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:19.591748 24926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:19.594762 24926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "bd780c16311c4b94b30e6cde4181768d"
format_stamp: "Formatted at 2026-06-19 06:45:19 on dist-test-slave-5w2x"
server_key: "4ba01043951473d56fc8ad912ad017e6"
server_key_iv: "296be71c1379e22cb25a344a11006692"
server_key_version: "encryptionkey@0"
I20260619 06:45:19.595546 24926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "bd780c16311c4b94b30e6cde4181768d"
format_stamp: "Formatted at 2026-06-19 06:45:19 on dist-test-slave-5w2x"
server_key: "4ba01043951473d56fc8ad912ad017e6"
server_key_iv: "296be71c1379e22cb25a344a11006692"
server_key_version: "encryptionkey@0"
I20260619 06:45:19.600821 24926 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 06:45:19.604461 24941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:19.605818 24926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260619 06:45:19.606022 24926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "bd780c16311c4b94b30e6cde4181768d"
format_stamp: "Formatted at 2026-06-19 06:45:19 on dist-test-slave-5w2x"
server_key: "4ba01043951473d56fc8ad912ad017e6"
server_key_iv: "296be71c1379e22cb25a344a11006692"
server_key_version: "encryptionkey@0"
I20260619 06:45:19.606226 24926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:19.625725 24926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:19.630419 24926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:19.630718 24926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:19.649277 24926 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:36357
I20260619 06:45:19.649317 24992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:36357 every 8 connection(s)
I20260619 06:45:19.650664 24926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:45:19.653071 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 24926
I20260619 06:45:19.653277 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:45:19.654093 16701 external_mini_cluster.cc:1428] Setting key 618a3a69bf3e59ff45e287bb00fa3dcc
I20260619 06:45:19.655280 24993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 06:45:19 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:45:19.669633 24993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d: Bootstrap starting.
I20260619 06:45:19.673496 24993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:19.674539 24993 log.cc:824] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:19.677202 24996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:19.656111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46974 (local address 127.16.79.126:36357)
0619 06:45:19.656937 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:19.656948 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:19.656987 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:45:19.658514 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:19.658536 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:19.658944 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:19.659276 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:19.661132 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:19.662459 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:19.664206 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:19.664883 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:19.665018 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:19.669587 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:19.669595 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:19.669609 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:19.669654 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:19.673166 (+ 3512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:19.673923 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:19.673929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:19.673936 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:19.674094 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:19.674485 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:19.674490 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:19.674492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:19.675049 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:19.675372 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:19.675793 (+ 421us) server_negotiation.cc:299] Negotiation successful
0619 06:45:19.676256 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":364,"threads_started":1}
I20260619 06:45:19.677557 24993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d: No bootstrap required, opened a new log
I20260619 06:45:19.685678 24993 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd780c16311c4b94b30e6cde4181768d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36357 } }
I20260619 06:45:19.686103 24993 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:19.686215 24993 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd780c16311c4b94b30e6cde4181768d, State: Initialized, Role: FOLLOWER
I20260619 06:45:19.686792 24993 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd780c16311c4b94b30e6cde4181768d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36357 } }
I20260619 06:45:19.687032 24993 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:45:19.687188 24993 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:45:19.687359 24993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:19.689594 24993 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd780c16311c4b94b30e6cde4181768d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36357 } }
I20260619 06:45:19.690142 24993 leader_election.cc:302] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd780c16311c4b94b30e6cde4181768d; no voters:
I20260619 06:45:19.690709 24993 leader_election.cc:288] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:45:19.690862 24998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:45:19.692127 24998 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 1 LEADER]: Becoming Leader. State: Replica: bd780c16311c4b94b30e6cde4181768d, State: Running, Role: LEADER
I20260619 06:45:19.692667 24998 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd780c16311c4b94b30e6cde4181768d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36357 } }
I20260619 06:45:19.692973 24993 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:45:19.695227 24999 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd780c16311c4b94b30e6cde4181768d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd780c16311c4b94b30e6cde4181768d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36357 } } }
I20260619 06:45:19.695354 25000 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd780c16311c4b94b30e6cde4181768d. Latest consensus state: current_term: 1 leader_uuid: "bd780c16311c4b94b30e6cde4181768d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd780c16311c4b94b30e6cde4181768d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36357 } } }
I20260619 06:45:19.695757 25000 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:19.695632 24999 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:19.696938 25005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:45:19.703020 25005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:45:19.712643 25005 catalog_manager.cc:1347] Generated new cluster ID: f858ba750fde426e8f1eebf90281cdbc
I20260619 06:45:19.712796 25005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:45:19.734242 25005 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:45:19.735646 25005 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:45:19.741374 25005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d: Generated new TSK 0
I20260619 06:45:19.742208 25005 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:45:19.795099 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:36357
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:20.145915 25019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:20.146193 25019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:20.146296 25019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:20.153528 25019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:20.153643 25019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:20.153760 25019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:45:20.163406 25019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:36357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:20.164705 25019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:20.166019 25019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:20.179414 25027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:20.179557 25024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:20.180284 25025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:20.180444 25019 server_base.cc:1034] running on GCE node
I20260619 06:45:20.181109 25019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:20.182071 25019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:20.183317 25019 hybrid_clock.cc:648] HybridClock initialized: now 1781851520183261 us; error 50 us; skew 500 ppm
I20260619 06:45:20.183641 25019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:20 dist-test-slave-5w2x krb5kdc[24913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851520, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:20.186913 25019 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:45:20.188511 25019 webserver.cc:460] Webserver started at http://127.16.79.65:45985/ using document root <none> and password file <none>
I20260619 06:45:20.189211 25019 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:20.189360 25019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:20.189683 25019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:20.192744 25019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "ba3653d77f964fddab4028164ea155d2"
format_stamp: "Formatted at 2026-06-19 06:45:20 on dist-test-slave-5w2x"
server_key: "7a346c06fa28b90eb4a734e65fcb2655"
server_key_iv: "9a06a4820c57e086416270a29d36a33f"
server_key_version: "encryptionkey@0"
I20260619 06:45:20.193568 25019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "ba3653d77f964fddab4028164ea155d2"
format_stamp: "Formatted at 2026-06-19 06:45:20 on dist-test-slave-5w2x"
server_key: "7a346c06fa28b90eb4a734e65fcb2655"
server_key_iv: "9a06a4820c57e086416270a29d36a33f"
server_key_version: "encryptionkey@0"
I20260619 06:45:20.198779 25019 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:45:20.202347 25034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:20.203835 25019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:45:20.204000 25019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "ba3653d77f964fddab4028164ea155d2"
format_stamp: "Formatted at 2026-06-19 06:45:20 on dist-test-slave-5w2x"
server_key: "7a346c06fa28b90eb4a734e65fcb2655"
server_key_iv: "9a06a4820c57e086416270a29d36a33f"
server_key_version: "encryptionkey@0"
I20260619 06:45:20.204272 25019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:20.229096 25019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:20.234000 25019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:20.234323 25019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:20.235414 25019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:20.236960 25019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:20.237037 25019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:20.237128 25019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:20.237172 25019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:20.270350 25019 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:34593
I20260619 06:45:20.270455 25146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:34593 every 8 connection(s)
I20260619 06:45:20.271694 25019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:45:20.276737 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 25019
I20260619 06:45:20.276948 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:45:20.277568 16701 external_mini_cluster.cc:1428] Setting key 501e462cd00293249e8d1ecc75e10c7f
Jun 19 06:45:20 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851520, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:45:20.293816 25149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:20.274274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:58599 (local address 127.16.79.126:36357)
0619 06:45:20.274717 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:20.274730 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:20.275764 (+ 1034us) server_negotiation.cc:407] Connection header received
0619 06:45:20.277907 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:20.277913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:20.278002 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:20.278130 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:20.280229 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:20.281384 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:20.283000 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:20.283689 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:20.283841 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:20.287546 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:20.287551 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:20.287555 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:20.287588 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:20.290220 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:20.291240 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:20.291244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:20.291247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:20.291388 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:20.291919 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:20.291922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:20.291924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:20.292254 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:20.292379 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:20.293319 (+ 940us) server_negotiation.cc:299] Negotiation successful
0619 06:45:20.293512 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":198,"threads_started":1}
I20260619 06:45:20.295495 25150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:20.274858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:58599)
0619 06:45:20.275599 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:20.275638 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:20.277563 (+ 1925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:20.278495 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:20.278517 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:20.279275 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:20.280037 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:20.280055 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:20.281548 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:20.281552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:20.282841 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:20.282852 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:20.283914 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:20.283921 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:20.284230 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:20.284948 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:20.284971 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:20.287315 (+ 2344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:20.290383 (+ 3068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:20.290395 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:20.290418 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:20.291090 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:20.291515 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:20.291522 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:20.291529 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:20.291800 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:20.292384 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:20.292395 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:20.292871 (+ 476us) client_negotiation.cc:769] Sending connection context
0619 06:45:20.293214 (+ 343us) client_negotiation.cc:240] Negotiation successful
0619 06:45:20.293850 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":283,"threads_started":1}
I20260619 06:45:20.297199 25147 heartbeater.cc:347] Connected to a master server at 127.16.79.126:36357
I20260619 06:45:20.297554 25147 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:20.298372 25147 heartbeater.cc:511] Master 127.16.79.126:36357 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
I20260619 06:45:20.300655 24958 ts_manager.cc:194] Registered new tserver with Master: ba3653d77f964fddab4028164ea155d2 (127.16.79.65:34593)
I20260619 06:45:20.302235 24958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:58599
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:45:20.331157 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:36357
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:20.665529 25153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:20.665823 25153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:20.665922 25153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:20.673333 25153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:20.673445 25153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:20.673563 25153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:45:20.683540 25153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:36357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:20.684898 25153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:20.686251 25153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:20.699576 25159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:20.699615 25158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:20.700223 25161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:20.700899 25153 server_base.cc:1034] running on GCE node
I20260619 06:45:20.701561 25153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:20.702695 25153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:20.703970 25153 hybrid_clock.cc:648] HybridClock initialized: now 1781851520703916 us; error 54 us; skew 500 ppm
I20260619 06:45:20.704432 25153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:20 dist-test-slave-5w2x krb5kdc[24913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851520, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:20.707784 25153 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:45:20.710291 25153 webserver.cc:460] Webserver started at http://127.16.79.66:42465/ using document root <none> and password file <none>
I20260619 06:45:20.711122 25153 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:20.711231 25153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:20.711503 25153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:20.714588 25153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "1328001329f3428eb64336ea36f0db4f"
format_stamp: "Formatted at 2026-06-19 06:45:20 on dist-test-slave-5w2x"
server_key: "29cbb56730cd19b0ecee548d4fc77f43"
server_key_iv: "3332904a11ee930d16aac719373ad7cc"
server_key_version: "encryptionkey@0"
I20260619 06:45:20.715376 25153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "1328001329f3428eb64336ea36f0db4f"
format_stamp: "Formatted at 2026-06-19 06:45:20 on dist-test-slave-5w2x"
server_key: "29cbb56730cd19b0ecee548d4fc77f43"
server_key_iv: "3332904a11ee930d16aac719373ad7cc"
server_key_version: "encryptionkey@0"
I20260619 06:45:20.721035 25153 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:45:20.725003 25168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:20.726508 25153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:45:20.726711 25153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "1328001329f3428eb64336ea36f0db4f"
format_stamp: "Formatted at 2026-06-19 06:45:20 on dist-test-slave-5w2x"
server_key: "29cbb56730cd19b0ecee548d4fc77f43"
server_key_iv: "3332904a11ee930d16aac719373ad7cc"
server_key_version: "encryptionkey@0"
I20260619 06:45:20.726929 25153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:20.751633 25153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:20.756366 25153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:20.756675 25153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:20.757704 25153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:20.759058 25153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:20.759132 25153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:20.759303 25153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:20.759356 25153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:20.793107 25153 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:36843
I20260619 06:45:20.793185 25280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:36843 every 8 connection(s)
I20260619 06:45:20.794555 25153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:45:20.803980 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 25153
I20260619 06:45:20.804173 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:45:20.804612 16701 external_mini_cluster.cc:1428] Setting key 03e19f4d1ae7339ac6c47ea765ed5569
Jun 19 06:45:20 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851520, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:45:20.816363 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:20.797305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:51217 (local address 127.16.79.126:36357)
0619 06:45:20.797772 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:20.797776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:20.798646 (+ 870us) server_negotiation.cc:407] Connection header received
0619 06:45:20.800557 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:20.800563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:20.800658 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:20.800783 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:20.802323 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:20.803054 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:20.804781 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:20.805230 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:20.805372 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:20.809203 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:20.809212 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:20.809219 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:20.809287 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:20.812962 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:20.813799 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:20.813807 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:20.813813 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:20.813986 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:20.814549 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:20.814552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:20.814554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:20.814822 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:20.814940 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:20.815854 (+ 914us) server_negotiation.cc:299] Negotiation successful
0619 06:45:20.816021 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":253,"threads_started":1}
I20260619 06:45:20.818089 25284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:20.797739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:51217)
0619 06:45:20.798490 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:20.798529 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:20.800272 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:20.801026 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:20.801036 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:20.801572 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:20.802141 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:20.802157 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:20.803216 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:20.803221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:20.804626 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:20.804638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:20.805459 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:20.805467 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:20.805719 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:20.806397 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:20.806420 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:20.808930 (+ 2510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:20.813143 (+ 4213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:20.813150 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:20.813167 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:20.813646 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:20.814115 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:20.814121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:20.814127 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:20.814416 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:20.814964 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:20.814974 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:20.815466 (+ 492us) client_negotiation.cc:769] Sending connection context
0619 06:45:20.815795 (+ 329us) client_negotiation.cc:240] Negotiation successful
0619 06:45:20.816440 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":261,"threads_started":1}
I20260619 06:45:20.819638 25281 heartbeater.cc:347] Connected to a master server at 127.16.79.126:36357
I20260619 06:45:20.819970 25281 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:20.820768 25281 heartbeater.cc:511] Master 127.16.79.126:36357 requested a full tablet report, sending...
I20260619 06:45:20.822271 24958 ts_manager.cc:194] Registered new tserver with Master: 1328001329f3428eb64336ea36f0db4f (127.16.79.66:36843)
I20260619 06:45:20.823117 24958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:51217
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:45:20.854890 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:36357
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:21.183172 25287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:21.183496 25287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:21.183660 25287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:21.190963 25287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:21.191116 25287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:21.191285 25287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:45:21.201489 25287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:36357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:21.202890 25287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:21.204313 25287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:21.217882 25292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:21.218066 25293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:21.218127 25295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:21.218930 25287 server_base.cc:1034] running on GCE node
I20260619 06:45:21.219520 25287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:21.220541 25287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:21.221757 25287 hybrid_clock.cc:648] HybridClock initialized: now 1781851521221712 us; error 42 us; skew 500 ppm
I20260619 06:45:21.222061 25287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:21 dist-test-slave-5w2x krb5kdc[24913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851521, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:21.225256 25287 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:45:21.226679 25287 webserver.cc:460] Webserver started at http://127.16.79.67:33459/ using document root <none> and password file <none>
I20260619 06:45:21.227303 25287 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:21.227398 25287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:21.227643 25287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:21.230492 25287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "56584097166d4329aafa7a1a37938a44"
format_stamp: "Formatted at 2026-06-19 06:45:21 on dist-test-slave-5w2x"
server_key: "b98fd4df26db504a348c641fb869cd52"
server_key_iv: "3d838f3e1d52e81a357cdd046abc5e27"
server_key_version: "encryptionkey@0"
I20260619 06:45:21.231204 25287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "56584097166d4329aafa7a1a37938a44"
format_stamp: "Formatted at 2026-06-19 06:45:21 on dist-test-slave-5w2x"
server_key: "b98fd4df26db504a348c641fb869cd52"
server_key_iv: "3d838f3e1d52e81a357cdd046abc5e27"
server_key_version: "encryptionkey@0"
I20260619 06:45:21.236315 25287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260619 06:45:21.239837 25302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:21.241334 25287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:45:21.241535 25287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "56584097166d4329aafa7a1a37938a44"
format_stamp: "Formatted at 2026-06-19 06:45:21 on dist-test-slave-5w2x"
server_key: "b98fd4df26db504a348c641fb869cd52"
server_key_iv: "3d838f3e1d52e81a357cdd046abc5e27"
server_key_version: "encryptionkey@0"
I20260619 06:45:21.241756 25287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:21.257929 25287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:21.262612 25287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:21.262943 25287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:21.264187 25287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:21.266021 25287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:21.266131 25287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:21.266283 25287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:21.266350 25287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:21.301359 25287 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:36599
I20260619 06:45:21.301414 25414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:36599 every 8 connection(s)
I20260619 06:45:21.302731 25287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:45:21.305755 25147 heartbeater.cc:503] Master 127.16.79.126:36357 was elected leader, sending a full tablet report...
I20260619 06:45:21.306195 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 25287
I20260619 06:45:21.306378 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:45:21.306983 16701 external_mini_cluster.cc:1428] Setting key 93a5fef50cf17a601ea64e359243e778
Jun 19 06:45:21 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851521, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:45:21.323751 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:21.305582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:41073 (local address 127.16.79.126:36357)
0619 06:45:21.305796 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:21.305804 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:21.307292 (+ 1488us) server_negotiation.cc:407] Connection header received
0619 06:45:21.309592 (+ 2300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:21.309597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:21.309714 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:21.309830 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:21.311465 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:21.312293 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:21.313884 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:21.314287 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:21.314431 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:21.317975 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:21.317982 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:21.317986 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:21.318030 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:21.320691 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:21.321421 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:21.321427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:21.321431 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:21.321563 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:21.322008 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:21.322013 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:21.322016 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:21.322311 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:21.322477 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:21.323167 (+ 690us) server_negotiation.cc:299] Negotiation successful
0619 06:45:21.323401 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:45:21.325268 25417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:21.306215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:41073)
0619 06:45:21.307126 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:21.307179 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:21.309328 (+ 2149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:21.310087 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:21.310096 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:21.310629 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:21.311298 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:21.311319 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:21.312491 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:21.312495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:21.313735 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:21.313747 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:21.314527 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:21.314535 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:21.314773 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:21.315478 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:21.315500 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:21.317716 (+ 2216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:21.320841 (+ 3125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:21.320851 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:21.320870 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:21.321299 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:21.321674 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:21.321679 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:21.321683 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:21.321888 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:21.322440 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:21.322449 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:21.322817 (+ 368us) client_negotiation.cc:769] Sending connection context
0619 06:45:21.323083 (+ 266us) client_negotiation.cc:240] Negotiation successful
0619 06:45:21.323634 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":355,"threads_started":1}
I20260619 06:45:21.327209 25415 heartbeater.cc:347] Connected to a master server at 127.16.79.126:36357
I20260619 06:45:21.327540 25415 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:21.328364 25415 heartbeater.cc:511] Master 127.16.79.126:36357 requested a full tablet report, sending...
I20260619 06:45:21.329797 24958 ts_manager.cc:194] Registered new tserver with Master: 56584097166d4329aafa7a1a37938a44 (127.16.79.67:36599)
I20260619 06:45:21.330626 24958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:41073
I20260619 06:45:21.334667 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.65.keytab.
Entry for principal oryx/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.65.keytab.
WARNING: no policy specified for oryx/127.16.79.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.79.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.66.keytab.
Entry for principal oryx/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.66.keytab.
WARNING: no policy specified for oryx/127.16.79.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.16.79.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.67.keytab.
Entry for principal oryx/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.67.keytab.
I20260619 06:45:21.467182 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 25019
Jun 19 06:45:21 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:45:21.483855 25425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:21.468428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57006 (local address 127.16.79.65:34593)
0619 06:45:21.469049 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:21.469055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:21.469082 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:45:21.469141 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:21.469148 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:21.469376 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:21.469530 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:21.470811 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:21.471842 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:21.473185 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:21.474019 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:21.474130 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:21.478090 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:21.478103 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:21.478118 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:21.478157 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:21.481066 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:21.481724 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:21.481731 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:21.481734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:21.481872 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:21.482222 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:21.482226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:21.482229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:21.482613 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:21.482915 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:21.483241 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 06:45:21.483443 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":313,"threads_started":1}
I20260619 06:45:21.485232 25142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:21.569211 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 25019
I20260619 06:45:21.585572 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 25153
Jun 19 06:45:21 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:45:21.601789 25427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:21.586447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57354 (local address 127.16.79.66:36843)
0619 06:45:21.586984 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:21.586991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:21.587018 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:45:21.587081 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:21.587087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:21.587296 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:21.587443 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:21.588741 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:21.589725 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:21.590953 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:21.591490 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:21.591586 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:21.595878 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:21.595891 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:21.595906 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:21.595955 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:21.598905 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:21.599624 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:21.599631 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:21.599634 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:21.599768 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:21.600084 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:21.600087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:21.600089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:21.600687 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:21.600918 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:21.601258 (+ 340us) server_negotiation.cc:299] Negotiation successful
0619 06:45:21.601488 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":271,"threads_started":1}
I20260619 06:45:21.603121 25276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:21.691550 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 25153
I20260619 06:45:21.707464 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 25287
Jun 19 06:45:21 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:45:21.723454 25429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:21.708596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52814 (local address 127.16.79.67:36599)
0619 06:45:21.709286 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:21.709292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:21.709323 (+ 31us) server_negotiation.cc:407] Connection header received
0619 06:45:21.709383 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:21.709389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:21.709595 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:21.709730 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:21.711093 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:21.712106 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:21.713444 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:21.713973 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:21.714070 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:21.718366 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:21.718378 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:21.718392 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:21.718426 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:21.721037 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:21.721738 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:21.721744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:21.721746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:21.721874 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:21.722164 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:21.722167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:21.722169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:21.722498 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:21.722767 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:21.722995 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 06:45:21.723173 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":360,"threads_started":1}
I20260619 06:45:21.724830 25410 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:21.807679 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 25287
I20260619 06:45:21.823298 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 24926
I20260619 06:45:21.824508 24988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:21.926857 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 24926
I20260619 06:45:21.947188 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:36357
--webserver_interface=127.16.79.126
--webserver_port=33929
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:36357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:22.276916 25432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:22.277197 25432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:22.277350 25432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:22.284607 25432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:45:22.284709 25432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:22.284811 25432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:22.284909 25432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:45:22.284988 25432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:45:22.294863 25432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:36357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:36357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=33929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:22.296273 25432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:22.297639 25432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:22.309283 25437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:22.309316 25438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:22.309486 25440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:22.310093 25432 server_base.cc:1034] running on GCE node
I20260619 06:45:22.310891 25432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:22.312352 25432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:22.313583 25432 hybrid_clock.cc:648] HybridClock initialized: now 1781851522313542 us; error 49 us; skew 500 ppm
I20260619 06:45:22.313946 25432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:22 dist-test-slave-5w2x krb5kdc[24913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851522, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:22.316957 25432 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:45:22.318427 25432 webserver.cc:460] Webserver started at http://127.16.79.126:33929/ using document root <none> and password file <none>
I20260619 06:45:22.319108 25432 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:22.319259 25432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:22.324680 25432 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:45:22.327636 25447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:22.329453 25432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:45:22.329636 25432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "bd780c16311c4b94b30e6cde4181768d"
format_stamp: "Formatted at 2026-06-19 06:45:19 on dist-test-slave-5w2x"
server_key: "4ba01043951473d56fc8ad912ad017e6"
server_key_iv: "296be71c1379e22cb25a344a11006692"
server_key_version: "encryptionkey@0"
I20260619 06:45:22.330938 25432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:22.354912 25432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:22.359230 25432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:22.359448 25432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:22.377189 25432 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:36357
I20260619 06:45:22.377193 25498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:36357 every 8 connection(s)
I20260619 06:45:22.378727 25432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:45:22.385397 25499 sys_catalog.cc:263] Verifying existing consensus state
I20260619 06:45:22.388471 25499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d: Bootstrap starting.
I20260619 06:45:22.388963 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 25432
I20260619 06:45:22.389740 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:34593
--local_ip_for_outbound_sockets=127.16.79.65
--tserver_master_addrs=127.16.79.126:36357
--webserver_port=45985
--webserver_interface=127.16.79.65
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.79.65
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:45:22.411880 25499 log.cc:824] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:22.423019 25499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:45:22.423614 25499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d: Bootstrap complete.
I20260619 06:45:22.437110 25499 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd780c16311c4b94b30e6cde4181768d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36357 } }
I20260619 06:45:22.438233 25499 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd780c16311c4b94b30e6cde4181768d, State: Initialized, Role: FOLLOWER
I20260619 06:45:22.438911 25499 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd780c16311c4b94b30e6cde4181768d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36357 } }
I20260619 06:45:22.439173 25499 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:45:22.439284 25499 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:45:22.439455 25499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 1 FOLLOWER]: Advancing to term 2
I20260619 06:45:22.444908 25499 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd780c16311c4b94b30e6cde4181768d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36357 } }
I20260619 06:45:22.445504 25499 leader_election.cc:302] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd780c16311c4b94b30e6cde4181768d; no voters:
I20260619 06:45:22.446133 25499 leader_election.cc:288] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 06:45:22.446286 25503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 2 FOLLOWER]: Leader election won for term 2
I20260619 06:45:22.447607 25503 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [term 2 LEADER]: Becoming Leader. State: Replica: bd780c16311c4b94b30e6cde4181768d, State: Running, Role: LEADER
I20260619 06:45:22.448372 25499 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:45:22.448340 25503 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd780c16311c4b94b30e6cde4181768d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36357 } }
I20260619 06:45:22.450685 25505 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd780c16311c4b94b30e6cde4181768d. Latest consensus state: current_term: 2 leader_uuid: "bd780c16311c4b94b30e6cde4181768d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd780c16311c4b94b30e6cde4181768d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36357 } } }
I20260619 06:45:22.451021 25505 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:22.451522 25504 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bd780c16311c4b94b30e6cde4181768d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd780c16311c4b94b30e6cde4181768d" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36357 } } }
I20260619 06:45:22.451856 25504 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:22.461524 25511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:45:22.467584 25511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:45:22.469008 25511 catalog_manager.cc:1259] Loaded cluster ID: f858ba750fde426e8f1eebf90281cdbc
I20260619 06:45:22.469161 25511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:45:22.473093 25511 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:45:22.475047 25511 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bd780c16311c4b94b30e6cde4181768d: Loaded TSK: 0
I20260619 06:45:22.476696 25511 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 06:45:22.751009 25501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:22.751295 25501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:22.751433 25501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:22.758884 25501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:22.759001 25501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:22.759176 25501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:45:22.769390 25501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.65.keytab
--principal=oryx/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:34593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=45985
--tserver_master_addrs=127.16.79.126:36357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:22.770705 25501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:22.772004 25501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:22.786265 25525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:22.786311 25526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:22.787755 25528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:22.788479 25501 server_base.cc:1034] running on GCE node
I20260619 06:45:22.789108 25501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:22.790154 25501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:22.791383 25501 hybrid_clock.cc:648] HybridClock initialized: now 1781851522791330 us; error 48 us; skew 500 ppm
I20260619 06:45:22.791700 25501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:22 dist-test-slave-5w2x krb5kdc[24913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851522, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:22.794946 25501 init.cc:377] Logged in from keytab as oryx/127.16.79.65@KRBTEST.COM (short username oryx)
I20260619 06:45:22.796581 25501 webserver.cc:460] Webserver started at http://127.16.79.65:45985/ using document root <none> and password file <none>
I20260619 06:45:22.797266 25501 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:22.797412 25501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:22.802902 25501 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260619 06:45:22.805728 25535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:22.807197 25501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:45:22.807374 25501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "ba3653d77f964fddab4028164ea155d2"
format_stamp: "Formatted at 2026-06-19 06:45:20 on dist-test-slave-5w2x"
server_key: "7a346c06fa28b90eb4a734e65fcb2655"
server_key_iv: "9a06a4820c57e086416270a29d36a33f"
server_key_version: "encryptionkey@0"
I20260619 06:45:22.808382 25501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:22.828275 25501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:22.832693 25501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:22.832962 25501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:22.833904 25501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:22.835372 25501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:22.835496 25501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:22.835618 25501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:22.835707 25501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:22.868549 25501 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:34593
I20260619 06:45:22.868633 25647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:34593 every 8 connection(s)
I20260619 06:45:22.870002 25501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:45:22.875512 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 25501
I20260619 06:45:22.876297 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:36843
--local_ip_for_outbound_sockets=127.16.79.66
--tserver_master_addrs=127.16.79.126:36357
--webserver_port=42465
--webserver_interface=127.16.79.66
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.79.66
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:45:22 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:22 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 06:45:23.274536 25652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:23.274821 25652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:23.274968 25652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:23.282272 25652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:23.282393 25652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:23.282565 25652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:45:23.292337 25652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.66.keytab
--principal=oryx/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:36843
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=42465
--tserver_master_addrs=127.16.79.126:36357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:23.293754 25652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:23.295127 25652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:23.310142 25659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:23.310101 25658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:23.310935 25661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:23.311043 25652 server_base.cc:1034] running on GCE node
I20260619 06:45:23.311791 25652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:23.312973 25652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:23.314201 25652 hybrid_clock.cc:648] HybridClock initialized: now 1781851523314150 us; error 45 us; skew 500 ppm
I20260619 06:45:23.314553 25652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:23 dist-test-slave-5w2x krb5kdc[24913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851523, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:23.317919 25652 init.cc:377] Logged in from keytab as oryx/127.16.79.66@KRBTEST.COM (short username oryx)
I20260619 06:45:23.319707 25652 webserver.cc:460] Webserver started at http://127.16.79.66:42465/ using document root <none> and password file <none>
I20260619 06:45:23.320494 25652 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:23.320638 25652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:23.326427 25652 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 06:45:23.329748 25668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:23.332175 25652 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 06:45:23.332430 25652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "1328001329f3428eb64336ea36f0db4f"
format_stamp: "Formatted at 2026-06-19 06:45:20 on dist-test-slave-5w2x"
server_key: "29cbb56730cd19b0ecee548d4fc77f43"
server_key_iv: "3332904a11ee930d16aac719373ad7cc"
server_key_version: "encryptionkey@0"
I20260619 06:45:23.333979 25652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:23.376603 25652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:23.382853 25652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:23.383188 25652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:23.384490 25652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:23.386252 25652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:23.386406 25652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260619 06:45:23.386531 25652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:23.386626 25652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:23.427500 25652 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:36843
I20260619 06:45:23.427554 25780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:36843 every 8 connection(s)
I20260619 06:45:23.429049 25652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:45:23.432112 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 25652
I20260619 06:45:23.432992 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:36599
--local_ip_for_outbound_sockets=127.16.79.67
--tserver_master_addrs=127.16.79.126:36357
--webserver_port=33459
--webserver_interface=127.16.79.67
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.16.79.67
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:45:23 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:23 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 06:45:23.779125 25785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:23.779462 25785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:23.779630 25785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:23.787176 25785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:23.787334 25785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:23.787525 25785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:45:23.797430 25785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/krb5kdc/oryx_127.16.79.67.keytab
--principal=oryx/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:36599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=33459
--tserver_master_addrs=127.16.79.126:36357
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:23.798839 25785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:23.800117 25785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:23.817520 25792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:23.817520 25791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:23.825106 25795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:23.830193 25785 server_base.cc:1034] running on GCE node
I20260619 06:45:23.831081 25785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:23.832293 25785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:23.833544 25785 hybrid_clock.cc:648] HybridClock initialized: now 1781851523833465 us; error 63 us; skew 500 ppm
I20260619 06:45:23.833937 25785 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:23 dist-test-slave-5w2x krb5kdc[24913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851523, etypes {rep=17 tkt=17 ses=17}, oryx/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:23.837486 25785 init.cc:377] Logged in from keytab as oryx/127.16.79.67@KRBTEST.COM (short username oryx)
I20260619 06:45:23.843145 25785 webserver.cc:460] Webserver started at http://127.16.79.67:33459/ using document root <none> and password file <none>
I20260619 06:45:23.843931 25785 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:23.844101 25785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:23.849960 25785 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260619 06:45:23.852931 25801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:23.854460 25785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:45:23.854651 25785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "56584097166d4329aafa7a1a37938a44"
format_stamp: "Formatted at 2026-06-19 06:45:21 on dist-test-slave-5w2x"
server_key: "b98fd4df26db504a348c641fb869cd52"
server_key_iv: "3d838f3e1d52e81a357cdd046abc5e27"
server_key_version: "encryptionkey@0"
I20260619 06:45:23.855620 25785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:23.894368 25785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:23.898983 25785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:23.899266 25785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:23.900365 25785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:23.901836 25785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:23.901954 25785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:23.902086 25785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:23.902158 25785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:23.945786 25785 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:36599
I20260619 06:45:23.945829 25913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:36599 every 8 connection(s)
I20260619 06:45:23.947281 25785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:45:23.953992 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 25785
Jun 19 06:45:23 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:23.984086 25917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:23.955505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46984 (local address 127.16.79.126:36357)
0619 06:45:23.963977 (+ 8472us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:23.963986 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:23.964011 (+ 25us) server_negotiation.cc:407] Connection header received
0619 06:45:23.964083 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:23.964087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:23.964193 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:23.967369 (+ 3176us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:23.967640 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:23.968871 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:23.970842 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:23.971519 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:23.972367 (+ 848us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:23.974460 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:23.974471 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:23.974494 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:23.974540 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:23.978676 (+ 4136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:23.979362 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:23.979373 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:23.979386 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:23.979603 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:23.981146 (+ 1543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:23.981151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:23.981155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:23.981685 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:23.982795 (+ 1110us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:23.983004 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 06:45:23.983325 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8285,"thread_start_us":4830,"threads_started":1}
Jun 19 06:45:23 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 06:45:24.404641 25651 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.520s user 0.004s sys 0.010s
I20260619 06:45:24.407624 25651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:22.873076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:53427)
0619 06:45:22.873886 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:22.873929 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:22.875635 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:22.877311 (+ 1676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:22.877320 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:22.877804 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:22.878341 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:22.878357 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:22.880676 (+ 2319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:22.880681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:22.881955 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:22.881967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:22.883312 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:22.883319 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:22.883644 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:22.884446 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:22.884485 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:24.405921 (+1521436us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":342,"threads_started":1}
W20260619 06:45:24.408517 25650 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:22.872995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:53427 (local address 127.16.79.126:36357)
0619 06:45:22.873786 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:22.873801 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:22.874047 (+ 246us) server_negotiation.cc:407] Connection header received
0619 06:45:22.876328 (+ 2281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:22.876350 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:22.876810 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:22.877209 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:22.878534 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:22.880511 (+ 1977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:22.882105 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:22.883050 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:22.883187 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:24.408049 (+1524862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:53427: BlockingRecv error: recv got EOF from 127.16.79.65:53427 (error 108)
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":301,"threads_started":1}
W20260619 06:45:24.409801 25648 heartbeater.cc:644] Failed to heartbeat to 127.16.79.126:36357 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.79.126:36357: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Jun 19 06:45:24 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:24 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:24.428107 25651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:24.410760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:37503)
0619 06:45:24.411172 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:24.411201 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:24.411355 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:24.411883 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:24.411891 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:24.412652 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:24.412976 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:24.412994 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:24.415204 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:24.415212 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:24.417687 (+ 2475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:24.417713 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:24.419596 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:24.419606 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:24.419898 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:24.420948 (+ 1050us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:24.420984 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:24.427642 (+ 6658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233}
W20260619 06:45:24.429636 25650 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:24.411108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:37503 (local address 127.16.79.126:36357)
0619 06:45:24.411400 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:24.411409 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:24.411437 (+ 28us) server_negotiation.cc:407] Connection header received
0619 06:45:24.411521 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:24.411529 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:24.411639 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:24.411812 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:24.413225 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:24.414988 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:24.418000 (+ 3012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:24.419037 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:24.419178 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:24.429276 (+ 10098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:37503: BlockingRecv error: recv got EOF from 127.16.79.65:37503 (error 108)
Metrics: {"server-negotiator.queue_time_us":130}
Jun 19 06:45:24 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:24 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:24.452212 25651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:24.432578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:43059)
0619 06:45:24.435171 (+ 2593us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:24.435199 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:24.435405 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:24.436318 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:24.436326 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:24.436963 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:24.437262 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:24.437278 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:24.439842 (+ 2564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:24.439850 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:24.442240 (+ 2390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:24.442261 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:24.443715 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:24.443725 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:24.444071 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:24.445418 (+ 1347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:24.445453 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:24.451743 (+ 6290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2432}
W20260619 06:45:24.452533 25650 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:24.433041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:43059 (local address 127.16.79.126:36357)
0619 06:45:24.434971 (+ 1930us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:24.434981 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:24.435276 (+ 295us) server_negotiation.cc:407] Connection header received
0619 06:45:24.435743 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:24.435752 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:24.435875 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:24.435971 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:24.437631 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:24.439557 (+ 1926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:24.442488 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:24.443484 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:24.443673 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:24.452084 (+ 8411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:43059: BlockingRecv error: recv got EOF from 127.16.79.65:43059 (error 108)
Metrics: {"server-negotiator.queue_time_us":1754}
W20260619 06:45:24.452919 25648 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 06:45:24.877676 25784 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.413s user 0.009s sys 0.009s
W20260619 06:45:24.879482 25783 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:23.431883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:51093 (local address 127.16.79.126:36357)
0619 06:45:23.447368 (+ 15485us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:23.447380 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:23.449484 (+ 2104us) server_negotiation.cc:407] Connection header received
0619 06:45:23.453531 (+ 4047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:23.453540 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:23.453656 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:23.453795 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:23.456838 (+ 3043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:23.458232 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:23.461790 (+ 3558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:23.462496 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:23.462620 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:24.879158 (+1416538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.66:51093: BlockingRecv error: recv got EOF from 127.16.79.66:51093 (error 108)
Metrics: {"server-negotiator.queue_time_us":15254,"thread_start_us":15500,"threads_started":1}
I20260619 06:45:24.880609 25784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:23.432410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:51093)
0619 06:45:23.449120 (+ 16710us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:23.449197 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:23.453076 (+ 3879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:23.454213 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:23.454232 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:23.455335 (+ 1103us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:23.456557 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:23.456593 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:23.458432 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:23.458442 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:23.461545 (+ 3103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:23.461569 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:23.462830 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:23.462841 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:23.463299 (+ 458us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:23.464336 (+ 1037us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:23.464379 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:24.878956 (+1414577us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":16220,"thread_start_us":15900,"threads_started":1}
W20260619 06:45:24.881685 25781 heartbeater.cc:644] Failed to heartbeat to 127.16.79.126:36357 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.79.126:36357: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Jun 19 06:45:24 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:24 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:24.896168 25784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:24.882489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:54375)
0619 06:45:24.882727 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:24.882746 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:24.882884 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:24.883206 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:24.883211 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:24.883589 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:24.883742 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:24.883750 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:24.884887 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:24.884893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:24.885866 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:24.885878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:24.886584 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:24.886590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:24.886771 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:24.892054 (+ 5283us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:24.892072 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:24.895814 (+ 3742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106}
W20260619 06:45:24.896250 25783 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:24.882501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:54375 (local address 127.16.79.126:36357)
0619 06:45:24.882665 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:24.882671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:24.882819 (+ 148us) server_negotiation.cc:407] Connection header received
0619 06:45:24.883020 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:24.883023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:24.883080 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:24.883177 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:24.883872 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:24.884754 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:24.886008 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:24.886413 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:24.886554 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:24.896004 (+ 9450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.66:54375: BlockingRecv error: recv got EOF from 127.16.79.66:54375 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 19 06:45:24 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:24 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:24.909154 25784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:24.896858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:50533)
0619 06:45:24.897100 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:24.897142 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:24.897290 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:24.897648 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:24.897656 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:24.898241 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:24.898533 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:24.898549 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:24.899538 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:24.899547 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:24.902098 (+ 2551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:24.902123 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:24.902849 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:24.902855 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:24.903044 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:24.903926 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:24.903943 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:24.908676 (+ 4733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260619 06:45:24.909183 25783 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:24.897059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:50533 (local address 127.16.79.126:36357)
0619 06:45:24.897204 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:24.897208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:24.897223 (+ 15us) server_negotiation.cc:407] Connection header received
0619 06:45:24.897427 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:24.897430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:24.897483 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:24.897557 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:24.898692 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:24.899387 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:24.902297 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:24.902692 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:24.902827 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:24.908852 (+ 6025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.66:50533: BlockingRecv error: recv got EOF from 127.16.79.66:50533 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260619 06:45:24.909873 25781 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 06:45:25.110664 25916 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.131s user 0.002s sys 0.011s
W20260619 06:45:25.111913 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:23.952376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:55403 (local address 127.16.79.126:36357)
0619 06:45:23.960666 (+ 8290us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:23.960676 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:23.964530 (+ 3854us) server_negotiation.cc:407] Connection header received
0619 06:45:23.967369 (+ 2839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:23.967374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:23.967468 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:23.967555 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:23.970168 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:23.971379 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:23.976699 (+ 5320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:23.977382 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:23.977494 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:25.111674 (+1134180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.67:55403: BlockingRecv error: recv got EOF from 127.16.79.67:55403 (error 108)
Metrics: {"server-negotiator.queue_time_us":8101,"spinlock_wait_cycles":13056,"thread_start_us":11255,"threads_started":1}
I20260619 06:45:25.112679 25916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:23.951649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:55403)
0619 06:45:23.964349 (+ 12700us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:23.964418 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:23.967041 (+ 2623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:23.968317 (+ 1276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:23.968330 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:23.969099 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:23.969949 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:23.969981 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:23.971610 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:23.971617 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:23.976219 (+ 4602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:23.976238 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:23.977866 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:23.977873 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:23.978189 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:23.979503 (+ 1314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:23.979538 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:25.111434 (+1131896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":12315,"thread_start_us":12502,"threads_started":1}
W20260619 06:45:25.113785 25914 heartbeater.cc:644] Failed to heartbeat to 127.16.79.126:36357 (0 consecutive failures): Not authorized: Failed to ping master at 127.16.79.126:36357: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Jun 19 06:45:25 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:25 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:25.123548 25916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:25.114570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:48419)
0619 06:45:25.114779 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:25.114800 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:25.114952 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:25.115292 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:25.115297 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:25.115698 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:25.115858 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:25.115867 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:25.117141 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:25.117147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:25.118137 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:25.118149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:25.118864 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:25.118870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:25.119044 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:25.119684 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:25.119701 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:25.123206 (+ 3505us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260619 06:45:25.123667 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:25.114636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:48419 (local address 127.16.79.126:36357)
0619 06:45:25.114794 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:25.114800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:25.114872 (+ 72us) server_negotiation.cc:407] Connection header received
0619 06:45:25.115087 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:25.115092 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:25.115165 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:25.115261 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:25.116012 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:25.116986 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:25.118281 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:25.118692 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:25.118838 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:25.123431 (+ 4593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.67:48419: BlockingRecv error: recv got EOF from 127.16.79.67:48419 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 19 06:45:25 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:25 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:25.132663 25916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:25.124369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:38501)
0619 06:45:25.124507 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:25.124521 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:25.124636 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:25.125065 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:25.125071 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:25.125527 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:25.125682 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:25.125691 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:25.126680 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:25.126685 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:25.127645 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:25.127655 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:25.128434 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:25.128440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:25.128641 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:25.129232 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:25.129249 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:25.132369 (+ 3120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 06:45:25.132833 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:25.124493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:38501 (local address 127.16.79.126:36357)
0619 06:45:25.124786 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:25.124798 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:25.124817 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:45:25.124863 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:25.124866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:25.124929 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:25.125030 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:25.125821 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:25.126546 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:25.127784 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:25.128286 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:25.128438 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:25.132562 (+ 4124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.67:38501: BlockingRecv error: recv got EOF from 127.16.79.67:38501 (error 108)
Metrics: {"server-negotiator.queue_time_us":172}
W20260619 06:45:25.133172 25914 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 06:45:25 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:25 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:25.461723 25921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:25.453391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:46029)
0619 06:45:25.453838 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:25.453867 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:25.453972 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:25.454250 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:25.454253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:25.454634 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:25.454771 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:25.454779 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:25.455703 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:25.455708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:25.456778 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:25.456790 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:25.457421 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:25.457425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:25.457533 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:25.458055 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:25.458071 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:25.461502 (+ 3431us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":254,"threads_started":1}
W20260619 06:45:25.461913 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:25.453476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:46029 (local address 127.16.79.126:36357)
0619 06:45:25.453692 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:25.453697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:25.453936 (+ 239us) server_negotiation.cc:407] Connection header received
0619 06:45:25.454075 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:25.454079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:25.454139 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:25.454209 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:25.454888 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:25.455558 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:25.456895 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:25.457274 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:25.457371 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:25.461688 (+ 4317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:46029: BlockingRecv error: recv got EOF from 127.16.79.65:46029 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 19 06:45:25 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:25 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:25.918691 25922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:25.910444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:35785)
0619 06:45:25.911022 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:25.911041 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:25.911185 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:25.911448 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:25.911451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:25.911831 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:25.911985 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:25.911992 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:25.913068 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:25.913072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:25.914001 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:25.914010 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:25.914718 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:25.914722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:25.914832 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:25.915366 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:25.915383 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:25.918472 (+ 3089us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":304,"threads_started":1}
W20260619 06:45:25.918963 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:25.910514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:35785 (local address 127.16.79.126:36357)
0619 06:45:25.910659 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:25.910663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:25.911112 (+ 449us) server_negotiation.cc:407] Connection header received
0619 06:45:25.911270 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:25.911273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:25.911332 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:25.911411 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:25.912176 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:25.912916 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:25.914141 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:25.914572 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:25.914666 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:25.918682 (+ 4016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.66:35785: BlockingRecv error: recv got EOF from 127.16.79.66:35785 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 19 06:45:26 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:26 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:26.143923 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:26.133877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:51519)
0619 06:45:26.134685 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:26.134707 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:26.134902 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:26.135452 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:26.135458 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:26.136003 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:26.136230 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:26.136247 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:26.137352 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:26.137356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:26.138574 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:26.138587 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:26.139413 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:26.139418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:26.139543 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:26.140185 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:26.140203 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:26.143690 (+ 3487us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":507,"threads_started":1}
W20260619 06:45:26.144201 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:26.133954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:51519 (local address 127.16.79.126:36357)
0619 06:45:26.134208 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:26.134217 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:26.134817 (+ 600us) server_negotiation.cc:407] Connection header received
0619 06:45:26.135035 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:26.135049 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:26.135154 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:26.135315 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:26.136401 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:26.137216 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:26.138697 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:26.139246 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:26.139344 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:26.143909 (+ 4565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.67:51519: BlockingRecv error: recv got EOF from 127.16.79.67:51519 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 06:45:26 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:26 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:26.471956 25924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:26.462847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:36877)
0619 06:45:26.463396 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:26.463412 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:26.463551 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:26.463951 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:26.463956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:26.464502 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:26.464667 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:26.464676 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:26.465691 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:26.465695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:26.466616 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:26.466628 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:26.467318 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:26.467322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:26.467442 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:26.468053 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:26.468072 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:26.471684 (+ 3612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":318,"threads_started":1}
W20260619 06:45:26.472143 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:26.462888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:36877 (local address 127.16.79.126:36357)
0619 06:45:26.463080 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:26.463086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:26.463498 (+ 412us) server_negotiation.cc:407] Connection header received
0619 06:45:26.463691 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:26.463699 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:26.463779 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:26.463887 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:26.464795 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:26.465542 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:26.466757 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:26.467159 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:26.467250 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:26.471906 (+ 4656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:36877: BlockingRecv error: recv got EOF from 127.16.79.65:36877 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 06:45:26 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:26 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:26.928555 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:26.919613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:58973)
0619 06:45:26.920077 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:26.920093 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:26.920271 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:26.920578 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:26.920581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:26.920937 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:26.921107 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:26.921115 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:26.922122 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:26.922126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:26.923042 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:26.923052 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:26.923758 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:26.923763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:26.923921 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:26.924643 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:26.924660 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:26.928201 (+ 3541us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":272,"threads_started":1}
W20260619 06:45:26.928617 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:26.919716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:58973 (local address 127.16.79.126:36357)
0619 06:45:26.919977 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:26.919983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:26.920215 (+ 232us) server_negotiation.cc:407] Connection header received
0619 06:45:26.920392 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:26.920395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:26.920458 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:26.920538 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:26.921281 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:26.922005 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:26.923173 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:26.923591 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:26.923688 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:26.928401 (+ 4713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.66:58973: BlockingRecv error: recv got EOF from 127.16.79.66:58973 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Jun 19 06:45:27 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:27 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:27.154220 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:27.144969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:40483)
0619 06:45:27.145507 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:27.145523 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:27.145644 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:27.145995 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:27.145999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:27.146401 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:27.146591 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:27.146599 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:27.147655 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:27.147659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:27.148707 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:27.148720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:27.149538 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:27.149542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:27.149667 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:27.150299 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:27.150318 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:27.153979 (+ 3661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":315,"threads_started":1}
W20260619 06:45:27.154423 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:27.145095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:40483 (local address 127.16.79.126:36357)
0619 06:45:27.145388 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:27.145393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:27.145596 (+ 203us) server_negotiation.cc:407] Connection header received
0619 06:45:27.145750 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:27.145754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:27.145827 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:27.145905 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:27.146749 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:27.147508 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:27.148838 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:27.149367 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:27.149482 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:27.154188 (+ 4706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.67:40483: BlockingRecv error: recv got EOF from 127.16.79.67:40483 (error 108)
Metrics: {"server-negotiator.queue_time_us":185}
Jun 19 06:45:27 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:27 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:27.481840 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:27.473081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:47029)
0619 06:45:27.473595 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:27.473615 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:27.473779 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:27.474140 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:27.474146 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:27.474574 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:27.474788 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:27.474798 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:27.475790 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:27.475797 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:27.476721 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:27.476731 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:27.477445 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:27.477450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:27.477634 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:27.478195 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:27.478211 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:27.481586 (+ 3375us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":272,"threads_started":1}
W20260619 06:45:27.482072 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:27.473379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:47029 (local address 127.16.79.126:36357)
0619 06:45:27.473555 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:27.473561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:27.473736 (+ 175us) server_negotiation.cc:407] Connection header received
0619 06:45:27.473918 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:27.473924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:27.473999 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:27.474103 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:27.474933 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:27.475656 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:27.476872 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:27.477294 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:27.477433 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:27.481812 (+ 4379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:47029: BlockingRecv error: recv got EOF from 127.16.79.65:47029 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 19 06:45:27 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:27 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:27.938688 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:27.929588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:54231)
0619 06:45:27.930014 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:27.930030 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:27.930140 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:27.930439 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:27.930442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:27.930777 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:27.930947 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:27.930955 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:27.931962 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:27.931966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:27.932964 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:27.932976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:27.933837 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:27.933845 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:27.933998 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:27.934737 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:27.934760 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:27.938321 (+ 3561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":238,"threads_started":1}
W20260619 06:45:27.938925 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:27.929652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:54231 (local address 127.16.79.126:36357)
0619 06:45:27.929858 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:27.929864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:27.930106 (+ 242us) server_negotiation.cc:407] Connection header received
0619 06:45:27.930234 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:27.930237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:27.930298 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:27.930384 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:27.931065 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:27.931839 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:27.933093 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:27.933686 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:27.933802 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:27.938697 (+ 4895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.66:54231: BlockingRecv error: recv got EOF from 127.16.79.66:54231 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 06:45:28 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:28 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:28.164000 25929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:28.155110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:56903)
0619 06:45:28.155729 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:28.155749 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:28.155870 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:28.156213 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:28.156217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:28.156561 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:28.156741 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:28.156749 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:28.157646 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:28.157649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:28.158545 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:28.158554 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:28.159247 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:28.159251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:28.159391 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:28.160025 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:28.160041 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:28.163726 (+ 3685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":307,"threads_started":1}
W20260619 06:45:28.164083 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:28.155228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:56903 (local address 127.16.79.126:36357)
0619 06:45:28.155393 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:28.155397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:28.155827 (+ 430us) server_negotiation.cc:407] Connection header received
0619 06:45:28.155966 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:28.155970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:28.156028 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:28.156109 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:28.156856 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:28.157531 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:28.158697 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:28.159125 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:28.159220 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:28.163905 (+ 4685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.67:56903: BlockingRecv error: recv got EOF from 127.16.79.67:56903 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 06:45:28 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:28 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:28.491775 25930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:28.482885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:41675)
0619 06:45:28.483352 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:28.483368 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:28.483515 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:28.483857 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:28.483861 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:28.484346 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:28.484553 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:28.484568 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:28.485606 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:28.485614 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:28.486602 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:28.486613 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:28.487279 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:28.487284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:28.487468 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:28.488086 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:28.488103 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:28.491416 (+ 3313us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":257,"threads_started":1}
W20260619 06:45:28.491904 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:28.482968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:41675 (local address 127.16.79.126:36357)
0619 06:45:28.483212 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:28.483218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:28.483456 (+ 238us) server_negotiation.cc:407] Connection header received
0619 06:45:28.483639 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:28.483645 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:28.483713 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:28.483817 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:28.484706 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:28.485479 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:28.486743 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:28.487137 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:28.487269 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:28.491635 (+ 4366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:41675: BlockingRecv error: recv got EOF from 127.16.79.65:41675 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Jun 19 06:45:28 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:28 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
W20260619 06:45:28.949142 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:28.939875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:47627 (local address 127.16.79.126:36357)
0619 06:45:28.940055 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:28.940059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:28.940493 (+ 434us) server_negotiation.cc:407] Connection header received
0619 06:45:28.940622 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:28.940626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:28.940686 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:28.940765 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:28.941496 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:28.942221 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:28.943450 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:28.943885 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:28.944048 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:28.948974 (+ 4926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.66:47627: BlockingRecv error: recv got EOF from 127.16.79.66:47627 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 06:45:28.949155 25931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:28.939833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:47627)
0619 06:45:28.940393 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:28.940421 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:28.940529 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:28.940819 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:28.940822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:28.941197 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:28.941373 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:28.941381 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:28.942346 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:28.942350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:28.943331 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:28.943342 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:28.944078 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:28.944082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:28.944288 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:28.944834 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:28.944865 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:28.948796 (+ 3931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":313,"threads_started":1}
Jun 19 06:45:29 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:29 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:29.173970 25932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:29.164951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:39489)
0619 06:45:29.165476 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:29.165492 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:29.165641 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:29.166037 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:29.166042 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:29.166429 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:29.166584 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:29.166592 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:29.167595 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:29.167603 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:29.168637 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:29.168648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:29.169342 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:29.169347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:29.169549 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:29.170183 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:29.170201 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:29.173711 (+ 3510us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":304,"threads_started":1}
W20260619 06:45:29.174239 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:29.165024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:39489 (local address 127.16.79.126:36357)
0619 06:45:29.165251 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:29.165259 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:29.165580 (+ 321us) server_negotiation.cc:407] Connection header received
0619 06:45:29.165794 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:29.165798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:29.165857 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:29.165977 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:29.166728 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:29.167470 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:29.168785 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:29.169196 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:29.169330 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:29.173914 (+ 4584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.67:39489: BlockingRecv error: recv got EOF from 127.16.79.67:39489 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 19 06:45:29 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:29 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:29.500846 25933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:29.492803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:45041)
0619 06:45:29.493318 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:29.493335 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:29.493448 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:29.493733 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:29.493736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:29.494168 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:29.494324 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:29.494332 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:29.495241 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:29.495244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:29.496175 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:29.496185 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:29.496783 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:29.496787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:29.496918 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:29.497442 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:29.497457 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:29.500624 (+ 3167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":284,"threads_started":1}
W20260619 06:45:29.501027 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:29.492896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:45041 (local address 127.16.79.126:36357)
0619 06:45:29.493052 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:29.493057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:29.493405 (+ 348us) server_negotiation.cc:407] Connection header received
0619 06:45:29.493543 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:29.493546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:29.493613 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:29.493690 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:29.494444 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:29.495119 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:29.496293 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:29.496668 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:29.496759 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:29.500821 (+ 4062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:45041: BlockingRecv error: recv got EOF from 127.16.79.65:45041 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 19 06:45:29 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:29 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:29.969064 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:29.950338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:52301)
0619 06:45:29.959115 (+ 8777us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:29.959132 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:29.959326 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:29.959694 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:29.959701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:29.960258 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:29.960470 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:29.960481 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:29.961624 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:29.961629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:29.962965 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:29.962983 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:29.963685 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:29.963689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:29.963818 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:29.964479 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:29.964545 (+ 66us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:29.968739 (+ 4194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":8656,"thread_start_us":8534,"threads_started":1}
W20260619 06:45:29.969244 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:29.950466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:52301 (local address 127.16.79.126:36357)
0619 06:45:29.950661 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:29.950667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:29.959241 (+ 8574us) server_negotiation.cc:407] Connection header received
0619 06:45:29.959446 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:29.959452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:29.959557 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:29.959653 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:29.960599 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:29.961479 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:29.963101 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:29.963554 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:29.963658 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:29.968978 (+ 5320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.66:52301: BlockingRecv error: recv got EOF from 127.16.79.66:52301 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 19 06:45:30 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:30 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:30.183573 25935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:30.174903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:57157)
0619 06:45:30.175419 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:30.175445 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:30.175566 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:30.175848 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:30.175852 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:30.176294 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:30.176455 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:30.176463 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:30.177379 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:30.177382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:30.178296 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:30.178307 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:30.178935 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:30.178939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:30.179070 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:30.179626 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:30.179642 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:30.183365 (+ 3723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":284,"threads_started":1}
W20260619 06:45:30.183768 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:30.174992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:57157 (local address 127.16.79.126:36357)
0619 06:45:30.175148 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:30.175152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:30.175517 (+ 365us) server_negotiation.cc:407] Connection header received
0619 06:45:30.175661 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:30.175664 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:30.175724 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:30.175804 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:30.176576 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:30.177261 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:30.178426 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:30.178808 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:30.178908 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:30.183558 (+ 4650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.67:57157: BlockingRecv error: recv got EOF from 127.16.79.67:57157 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 06:45:30 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:30 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:30.510541 25936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:30.501846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:34343)
0619 06:45:30.502323 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:30.502340 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:30.502456 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:30.502736 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:30.502739 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:30.503099 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:30.503243 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:30.503251 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:30.504224 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:30.504227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:30.505136 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:30.505147 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:30.505856 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:30.505861 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:30.505987 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:30.506576 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:30.506594 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:30.510211 (+ 3617us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":248,"threads_started":1}
W20260619 06:45:30.510759 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:30.501895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:34343 (local address 127.16.79.126:36357)
0619 06:45:30.502137 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:30.502144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:30.502418 (+ 274us) server_negotiation.cc:407] Connection header received
0619 06:45:30.502552 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:30.502556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:30.502618 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:30.502698 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:30.503360 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:30.504066 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:30.505266 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:30.505686 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:30.505779 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:30.510492 (+ 4713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:34343: BlockingRecv error: recv got EOF from 127.16.79.65:34343 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 06:45:30 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:30 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:30.978945 25937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:30.970243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:59903)
0619 06:45:30.970810 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:30.970825 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:30.970992 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:30.971350 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:30.971354 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:30.971791 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:30.971938 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:30.971946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:30.973045 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:30.973051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:30.973993 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:30.974004 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:30.974711 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:30.974718 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:30.974903 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:30.975481 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:30.975497 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:30.978721 (+ 3224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":347,"threads_started":1}
W20260619 06:45:30.979153 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:30.970391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:59903 (local address 127.16.79.126:36357)
0619 06:45:30.970654 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:30.970661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:30.970939 (+ 278us) server_negotiation.cc:407] Connection header received
0619 06:45:30.971139 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:30.971149 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:30.971209 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:30.971318 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:30.972103 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:30.972891 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:30.974135 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:30.974554 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:30.974693 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:30.978918 (+ 4225us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.66:59903: BlockingRecv error: recv got EOF from 127.16.79.66:59903 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Jun 19 06:45:31 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:31 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:31.193061 25938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:31.184454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:52287)
0619 06:45:31.184859 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:31.184876 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:31.184989 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:31.185275 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:31.185278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:31.185720 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:31.185882 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:31.185891 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:31.186918 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:31.186922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:31.187811 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:31.187822 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:31.188685 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:31.188690 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:31.188862 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:31.189482 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:31.189499 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:31.192837 (+ 3338us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":220,"threads_started":1}
W20260619 06:45:31.193259 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:31.184575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:52287 (local address 127.16.79.126:36357)
0619 06:45:31.184799 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:31.184805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:31.184957 (+ 152us) server_negotiation.cc:407] Connection header received
0619 06:45:31.185083 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:31.185087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:31.185150 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:31.185238 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:31.186012 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:31.186796 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:31.187963 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:31.188508 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:31.188651 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:31.193020 (+ 4369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.67:52287: BlockingRecv error: recv got EOF from 127.16.79.67:52287 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Jun 19 06:45:31 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:31 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:31.520618 25939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:31.511469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:36661)
0619 06:45:31.512172 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:31.512192 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:31.512337 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:31.512687 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:31.512692 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:31.513118 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:31.513297 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:31.513305 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:31.514538 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:31.514542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:31.515471 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:31.515482 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:31.516115 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:31.516119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:31.516247 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:31.516874 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:31.516891 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:31.520368 (+ 3477us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":271,"threads_started":1}
W20260619 06:45:31.520895 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:31.511554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:36661 (local address 127.16.79.126:36357)
0619 06:45:31.511725 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:31.511730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:31.512343 (+ 613us) server_negotiation.cc:407] Connection header received
0619 06:45:31.512480 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:31.512484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:31.512544 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:31.512633 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:31.513474 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:31.514413 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:31.515583 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:31.515995 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:31.516084 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:31.520610 (+ 4526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:36661: BlockingRecv error: recv got EOF from 127.16.79.65:36661 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 06:45:31 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:31 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:31.989190 25940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:31.980077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:44671)
0619 06:45:31.980999 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:31.981017 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:31.981132 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:31.981586 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:31.981590 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:31.982018 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:31.982196 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:31.982204 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:31.983175 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:31.983178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:31.984109 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:31.984119 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:31.984908 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:31.984914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:31.985087 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:31.985640 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:31.985658 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:31.988866 (+ 3208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":791,"thread_start_us":571,"threads_started":1}
W20260619 06:45:31.989323 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:31.980180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:44671 (local address 127.16.79.126:36357)
0619 06:45:31.980426 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:31.980433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:31.981097 (+ 664us) server_negotiation.cc:407] Connection header received
0619 06:45:31.981358 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:31.981362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:31.981424 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:31.981510 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:31.982364 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:31.983067 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:31.984295 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:31.984754 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:31.984893 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:31.989068 (+ 4175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.66:44671: BlockingRecv error: recv got EOF from 127.16.79.66:44671 (error 108)
Metrics: {"server-negotiator.queue_time_us":134}
Jun 19 06:45:32 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:32 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:32.203325 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:32.194058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:51487)
0619 06:45:32.194628 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:32.194644 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:32.194803 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:32.195137 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:32.195142 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:32.195603 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:32.195831 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:32.195845 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:32.197136 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:32.197141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:32.198130 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:32.198140 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:32.198774 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:32.198777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:32.198887 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:32.199531 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:32.199548 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:32.202993 (+ 3445us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":266,"threads_started":1}
W20260619 06:45:32.203374 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:32.194141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:51487 (local address 127.16.79.126:36357)
0619 06:45:32.194361 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:32.194368 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:32.194733 (+ 365us) server_negotiation.cc:407] Connection header received
0619 06:45:32.194946 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:32.194949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:32.195007 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:32.195110 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:32.195996 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:32.196996 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:32.198247 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:32.198650 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:32.198743 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:32.203164 (+ 4421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.67:51487: BlockingRecv error: recv got EOF from 127.16.79.67:51487 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Jun 19 06:45:32 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:32 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:32.531224 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:32.521692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:60559)
0619 06:45:32.522250 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:32.522266 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:32.522391 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:32.522733 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:32.522737 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:32.523169 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:32.523372 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:32.523387 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:32.524686 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:32.524691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:32.526065 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:32.526080 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:32.526785 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:32.526790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:32.526911 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:32.527519 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:32.527536 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:32.530873 (+ 3337us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":298,"threads_started":1}
W20260619 06:45:32.531378 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:32.521858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:60559 (local address 127.16.79.126:36357)
0619 06:45:32.522047 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:32.522051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:32.522344 (+ 293us) server_negotiation.cc:407] Connection header received
0619 06:45:32.522496 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:32.522500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:32.522560 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:32.522648 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:32.523508 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:32.524524 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:32.526217 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:32.526624 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:32.526724 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:32.531094 (+ 4370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:60559: BlockingRecv error: recv got EOF from 127.16.79.65:60559 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Jun 19 06:45:32 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:32 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:32.999061 25943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:32.990113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:35109)
0619 06:45:32.990620 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:32.990635 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:32.990740 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:32.991068 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:32.991070 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:32.991427 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:32.991600 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:32.991608 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:32.992752 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:32.992756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:32.993668 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:32.993678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:32.994308 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:32.994312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:32.994422 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:32.994990 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:32.995006 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:32.998693 (+ 3687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":258,"threads_started":1}
W20260619 06:45:32.999198 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:32.990175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:35109 (local address 127.16.79.126:36357)
0619 06:45:32.990368 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:32.990372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:32.990705 (+ 333us) server_negotiation.cc:407] Connection header received
0619 06:45:32.990858 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:32.990861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:32.990931 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:32.991026 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:32.991715 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:32.992628 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:32.993783 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:32.994166 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:32.994249 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:32.998947 (+ 4698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.66:35109: BlockingRecv error: recv got EOF from 127.16.79.66:35109 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 19 06:45:33 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:33 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:33.213127 25944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:33.204313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:39033)
0619 06:45:33.204853 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:33.204869 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:33.204989 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:33.205308 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:33.205311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:33.205700 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:33.205862 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:33.205871 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:33.206872 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:33.206875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:33.207795 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:33.207805 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:33.208580 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:33.208585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:33.208699 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:33.209337 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:33.209359 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:33.212860 (+ 3501us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":286,"threads_started":1}
W20260619 06:45:33.213286 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:33.204379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:39033 (local address 127.16.79.126:36357)
0619 06:45:33.204551 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:33.204556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:33.204978 (+ 422us) server_negotiation.cc:407] Connection header received
0619 06:45:33.205090 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:33.205094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:33.205168 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:33.205266 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:33.206000 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:33.206733 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:33.207935 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:33.208427 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:33.208523 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:33.213064 (+ 4541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.67:39033: BlockingRecv error: recv got EOF from 127.16.79.67:39033 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Jun 19 06:45:33 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:33 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:33.541620 25945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:33.532470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:40517)
0619 06:45:33.533008 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:33.533025 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:33.533131 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:33.533402 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:33.533405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:33.533804 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:33.533961 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:33.533970 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:33.535047 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:33.535051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:33.536087 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:33.536099 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:33.536858 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:33.536865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:33.537036 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:33.537652 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:33.537669 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:33.541330 (+ 3661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":267,"threads_started":1}
W20260619 06:45:33.541790 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:33.532587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:40517 (local address 127.16.79.126:36357)
0619 06:45:33.532763 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:33.532768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:33.533094 (+ 326us) server_negotiation.cc:407] Connection header received
0619 06:45:33.533222 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:33.533226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:33.533284 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:33.533365 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:33.534130 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:33.534932 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:33.536302 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:33.536699 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:33.536828 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:33.541529 (+ 4701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:40517: BlockingRecv error: recv got EOF from 127.16.79.65:40517 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 19 06:45:34 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:34 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:34.009215 25946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:34.000082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:35267)
0619 06:45:34.000652 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:34.000673 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:34.000817 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:34.001212 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:34.001216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:34.001727 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:34.001872 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:34.001881 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:34.002907 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:34.002913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:34.003853 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:34.003864 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:34.004652 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:34.004657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:34.004838 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:34.005390 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:34.005407 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:34.008883 (+ 3476us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":306,"threads_started":1}
W20260619 06:45:34.009341 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:34.000183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:35267 (local address 127.16.79.126:36357)
0619 06:45:34.000394 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:34.000403 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:34.000760 (+ 357us) server_negotiation.cc:407] Connection header received
0619 06:45:34.000977 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:34.000982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:34.001062 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:34.001187 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:34.002013 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:34.002764 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:34.003992 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:34.004500 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:34.004642 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:34.009089 (+ 4447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.66:35267: BlockingRecv error: recv got EOF from 127.16.79.66:35267 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 06:45:34 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:34 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:34.223237 25947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:34.214258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:42385)
0619 06:45:34.214882 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:34.214898 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:34.215019 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:34.215303 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:34.215306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:34.215678 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:34.215821 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:34.215829 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:34.216779 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:34.216782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:34.217703 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:34.217713 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:34.218651 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:34.218656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:34.218849 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:34.219541 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:34.219565 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:34.223020 (+ 3455us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":389,"threads_started":1}
W20260619 06:45:34.223416 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:34.214361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:42385 (local address 127.16.79.126:36357)
0619 06:45:34.214567 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:34.214572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:34.214980 (+ 408us) server_negotiation.cc:407] Connection header received
0619 06:45:34.215116 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:34.215119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:34.215177 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:34.215261 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:34.215951 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:34.216660 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:34.217851 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:34.218468 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:34.218592 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:34.223191 (+ 4599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.67:42385: BlockingRecv error: recv got EOF from 127.16.79.67:42385 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 06:45:34 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:34 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:34.551774 25948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:34.542678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:40637)
0619 06:45:34.543122 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:34.543139 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:34.543254 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:34.543575 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:34.543579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:34.543919 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:34.544067 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:34.544076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:34.545227 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:34.545231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:34.546276 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:34.546288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:34.546968 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:34.546973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:34.547095 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:34.547737 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:34.547755 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:34.551540 (+ 3785us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":248,"threads_started":1}
W20260619 06:45:34.551951 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:34.542736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:40637 (local address 127.16.79.126:36357)
0619 06:45:34.542966 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:34.542971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:34.543218 (+ 247us) server_negotiation.cc:407] Connection header received
0619 06:45:34.543354 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:34.543357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:34.543431 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:34.543515 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:34.544346 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:34.545072 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:34.546407 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:34.546808 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:34.546906 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:34.551716 (+ 4810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:40637: BlockingRecv error: recv got EOF from 127.16.79.65:40637 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 19 06:45:35 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:35 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:35.025352 25949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:35.010405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:55705)
0619 06:45:35.010926 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:35.010942 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:35.011068 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:35.011463 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:35.011466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:35.011817 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:35.011963 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:35.011972 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:35.013056 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:35.013060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:35.014150 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:35.014185 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:35.015147 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:35.015152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:35.015318 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:35.021176 (+ 5858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:35.021197 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:35.025073 (+ 3876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":287,"threads_started":1}
W20260619 06:45:35.025635 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:35.010490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:55705 (local address 127.16.79.126:36357)
0619 06:45:35.010671 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:35.010675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:35.011028 (+ 353us) server_negotiation.cc:407] Connection header received
0619 06:45:35.011198 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:35.011205 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:35.011325 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:35.011418 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:35.012101 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:35.012912 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:35.014305 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:35.014975 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:35.015118 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:35.025335 (+ 10217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.66:55705: BlockingRecv error: recv got EOF from 127.16.79.66:55705 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 19 06:45:35 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:35 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:35.232844 25950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:35.224212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:60227)
0619 06:45:35.224793 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:35.224809 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:35.224919 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:35.225195 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:35.225198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:35.225578 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:35.225729 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:35.225737 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:35.226695 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:35.226698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:35.227634 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:35.227645 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:35.228311 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:35.228315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:35.228445 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:35.229096 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:35.229113 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:35.232474 (+ 3361us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":341,"threads_started":1}
W20260619 06:45:35.232926 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:35.224328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:60227 (local address 127.16.79.126:36357)
0619 06:45:35.224523 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:35.224527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:35.224880 (+ 353us) server_negotiation.cc:407] Connection header received
0619 06:45:35.225014 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:35.225017 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:35.225074 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:35.225153 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:35.225853 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:35.226578 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:35.227751 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:35.228193 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:35.228282 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:35.232640 (+ 4358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.67:60227: BlockingRecv error: recv got EOF from 127.16.79.67:60227 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Jun 19 06:45:35 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:35 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:35.562003 25951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:35.552635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:56629)
0619 06:45:35.553072 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:35.553088 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:35.553201 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:35.553487 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:35.553490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:35.553867 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:35.554039 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:35.554047 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:35.555384 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:35.555389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:35.556349 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:35.556369 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:35.557082 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:35.557087 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:35.557213 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:35.557939 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:35.557957 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:35.561657 (+ 3700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":231,"threads_started":1}
W20260619 06:45:35.562193 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:35.552783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:56629 (local address 127.16.79.126:36357)
0619 06:45:35.553026 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:35.553033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:35.553164 (+ 131us) server_negotiation.cc:407] Connection header received
0619 06:45:35.553295 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:35.553299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:35.553366 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:35.553446 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:35.554156 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:35.555206 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:35.556498 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:35.556915 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:35.557016 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:35.561925 (+ 4909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:56629: BlockingRecv error: recv got EOF from 127.16.79.65:56629 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Jun 19 06:45:36 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:36 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:36.035166 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:36.026367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:50609)
0619 06:45:36.027037 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:36.027053 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:36.027157 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:36.027433 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:36.027436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:36.027811 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:36.027988 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:36.027996 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:36.029044 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:36.029048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:36.029954 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:36.029964 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:36.030578 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:36.030582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:36.030697 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:36.031255 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:36.031272 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:36.034835 (+ 3563us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":287,"threads_started":1}
W20260619 06:45:36.035238 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:36.026486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:50609 (local address 127.16.79.126:36357)
0619 06:45:36.026707 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:36.026713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:36.027131 (+ 418us) server_negotiation.cc:407] Connection header received
0619 06:45:36.027252 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:36.027256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:36.027316 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:36.027392 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:36.028104 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:36.028926 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:36.030068 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:36.030456 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:36.030545 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:36.035010 (+ 4465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.66:50609: BlockingRecv error: recv got EOF from 127.16.79.66:50609 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Jun 19 06:45:36 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:36 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:36.242233 25953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:36.233853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:53533)
0619 06:45:36.234320 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:36.234336 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:36.234493 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:36.234808 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:36.234812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:36.235242 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:36.235416 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:36.235424 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:36.236471 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:36.236478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:36.237377 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:36.237388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:36.238117 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:36.238122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:36.238303 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:36.238843 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:36.238860 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:36.242013 (+ 3153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":259,"threads_started":1}
W20260619 06:45:36.242405 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:36.233940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:53533 (local address 127.16.79.126:36357)
0619 06:45:36.234166 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:36.234173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:36.234464 (+ 291us) server_negotiation.cc:407] Connection header received
0619 06:45:36.234601 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:36.234607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:36.234676 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:36.234776 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:36.235554 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:36.236335 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:36.237515 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:36.237972 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:36.238103 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:36.242181 (+ 4078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.67:53533: BlockingRecv error: recv got EOF from 127.16.79.67:53533 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Jun 19 06:45:36 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:36 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:36.572430 25954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:36.563175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:54307)
0619 06:45:36.563658 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:36.563675 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:36.563788 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:36.564078 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:36.564081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:36.564476 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:36.564645 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:36.564653 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:36.565642 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:36.565646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:36.566584 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:36.566595 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:36.567280 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:36.567284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:36.567414 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:36.568012 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:36.568028 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:36.572050 (+ 4022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":269,"threads_started":1}
W20260619 06:45:36.572530 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:36.563190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:54307 (local address 127.16.79.126:36357)
0619 06:45:36.563469 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:36.563476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:36.563762 (+ 286us) server_negotiation.cc:407] Connection header received
0619 06:45:36.563889 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:36.563892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:36.563957 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:36.564037 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:36.564762 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:36.565529 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:36.566708 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:36.567152 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:36.567247 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:36.572274 (+ 5027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:54307: BlockingRecv error: recv got EOF from 127.16.79.65:54307 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Jun 19 06:45:37 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:37 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:37.045192 25955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:37.036104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:44605)
0619 06:45:37.036647 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:37.036664 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:37.036776 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:37.037043 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:37.037046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:37.037404 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:37.037591 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:37.037599 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:37.038701 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:37.038706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:37.039734 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:37.039746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:37.040532 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:37.040536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:37.040669 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:37.041441 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:37.041470 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:37.044870 (+ 3400us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":292,"threads_started":1}
W20260619 06:45:37.045334 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:37.036168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:44605 (local address 127.16.79.126:36357)
0619 06:45:37.036503 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:37.036509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:37.036740 (+ 231us) server_negotiation.cc:407] Connection header received
0619 06:45:37.036863 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:37.036867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:37.036927 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:37.037004 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:37.037841 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:37.038565 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:37.039902 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:37.040398 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:37.040497 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:37.045052 (+ 4555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.66:44605: BlockingRecv error: recv got EOF from 127.16.79.66:44605 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 06:45:37 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:37 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:37.252694 25956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:37.243377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:48845)
0619 06:45:37.243895 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:37.243914 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:37.244039 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:37.244380 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:37.244384 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:37.244778 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:37.244947 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:37.244971 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:37.245999 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:37.246003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:37.246966 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:37.246978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:37.247659 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:37.247663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:37.247789 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:37.248510 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:37.248539 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:37.252459 (+ 3920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":275,"threads_started":1}
W20260619 06:45:37.252979 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:37.243434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:48845 (local address 127.16.79.126:36357)
0619 06:45:37.243664 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:37.243673 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:37.244010 (+ 337us) server_negotiation.cc:407] Connection header received
0619 06:45:37.244192 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:37.244196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:37.244258 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:37.244339 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:37.245089 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:37.245842 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:37.247098 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:37.247494 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:37.247595 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:37.252674 (+ 5079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.67:48845: BlockingRecv error: recv got EOF from 127.16.79.67:48845 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Jun 19 06:45:37 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:37 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:37.582355 25957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:37.573351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:59283)
0619 06:45:37.573913 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:37.573930 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:37.574090 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:37.574450 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:37.574455 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:37.574848 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:37.575012 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:37.575020 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:37.576013 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:37.576018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:37.577202 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:37.577222 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:37.577944 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:37.577950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:37.578136 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:37.578717 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:37.578734 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:37.582121 (+ 3387us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":316,"threads_started":1}
W20260619 06:45:37.582545 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:37.573544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:59283 (local address 127.16.79.126:36357)
0619 06:45:37.573800 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:37.573806 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:37.574015 (+ 209us) server_negotiation.cc:407] Connection header received
0619 06:45:37.574218 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:37.574224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:37.574294 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:37.574401 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:37.575156 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:37.575864 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:37.577346 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:37.577796 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:37.577940 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:37.582316 (+ 4376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:59283: BlockingRecv error: recv got EOF from 127.16.79.65:59283 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Jun 19 06:45:38 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:38 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.66@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:38.055962 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:38.046476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.66:50161)
0619 06:45:38.046995 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:38.047012 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:38.047127 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:38.047409 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:38.047412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:38.047813 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:38.047961 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:38.047970 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:38.049406 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:38.049413 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:38.050641 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:38.050654 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:38.051362 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:38.051368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:38.051546 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:38.052156 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:38.052175 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:38.055595 (+ 3420us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":302,"threads_started":1}
W20260619 06:45:38.056082 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:38.046749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:50161 (local address 127.16.79.126:36357)
0619 06:45:38.046978 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:38.046985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:38.047093 (+ 108us) server_negotiation.cc:407] Connection header received
0619 06:45:38.047218 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:38.047222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:38.047280 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:38.047387 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:38.048190 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:38.049269 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:38.050778 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:38.051216 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:38.051350 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:38.055829 (+ 4479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.66:50161: BlockingRecv error: recv got EOF from 127.16.79.66:50161 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Jun 19 06:45:38 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:38 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.67@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:38.263105 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:38.253706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.67:52003)
0619 06:45:38.254175 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:38.254190 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:38.254353 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:38.254734 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:38.254739 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:38.255168 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:38.255314 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:38.255322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:38.256447 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:38.256452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:38.257355 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:38.257366 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:38.258087 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:38.258092 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:38.258276 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:38.258889 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:38.258912 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:38.262858 (+ 3946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":248,"threads_started":1}
W20260619 06:45:38.263350 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:38.253792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:52003 (local address 127.16.79.126:36357)
0619 06:45:38.254033 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:38.254040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:38.254284 (+ 244us) server_negotiation.cc:407] Connection header received
0619 06:45:38.254532 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:38.254536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:38.254598 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:38.254706 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:38.255466 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:38.256300 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:38.257514 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:38.257932 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:38.258061 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:38.263086 (+ 5025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.67:52003: BlockingRecv error: recv got EOF from 127.16.79.67:52003 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Jun 19 06:45:38 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
Jun 19 06:45:38 dist-test-slave-5w2x krb5kdc[24913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.16.79.65@KRBTEST.COM for oryx/127.16.79.126@KRBTEST.COM, Server not found in Kerberos database
I20260619 06:45:38.592486 25960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:38.583689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36357 (local address 127.16.79.65:46035)
0619 06:45:38.584255 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:38.584272 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:38.584380 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:38.584664 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:38.584667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:38.585008 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:38.585180 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:38.585187 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:38.586208 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:38.586212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:38.587170 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:38.587183 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:38.587887 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:38.587891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:38.588035 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:38.588705 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:38.588723 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:38.592125 (+ 3402us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.16.79.126:36357: Server oryx/127.16.79.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":283,"threads_started":1}
W20260619 06:45:38.592626 25919 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:38.583732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:46035 (local address 127.16.79.126:36357)
0619 06:45:38.584076 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:38.584083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:38.584352 (+ 269us) server_negotiation.cc:407] Connection header received
0619 06:45:38.584471 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:38.584475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:38.584535 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:38.584612 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:38.585320 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:38.586050 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:38.587303 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:38.587733 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:38.587822 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:38.592373 (+ 4551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.16.79.65:46035: BlockingRecv error: recv got EOF from 127.16.79.65:46035 (error 108)
Metrics: {"server-negotiator.queue_time_us":167}
I20260619 06:45:38.961066 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 25501
I20260619 06:45:38.973572 25962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:38.962156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53654 (local address 127.16.79.65:34593)
0619 06:45:38.962724 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:38.962731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:38.962760 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:45:38.962821 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:38.962827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:38.963087 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:38.963252 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:38.964744 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:38.965851 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:38.967089 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:38.967704 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:38.967812 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:38.970439 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:38.970448 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:38.970463 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:38.970509 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:38.973008 (+ 2499us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.79.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0619 06:45:38.973288 (+ 280us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53654: Request ticket server kudu/127.16.79.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":257,"threads_started":1}
W20260619 06:45:38.973688 25962 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53654: Request ticket server kudu/127.16.79.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260619 06:45:38.973729 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 25501: Not authorized: Client connection negotiation failed: client connection to 127.16.79.65:34593: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.79.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260619 06:45:38.974107 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 25501
I20260619 06:45:38.991415 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 25652
I20260619 06:45:39.003906 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:38.992619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45780 (local address 127.16.79.66:36843)
0619 06:45:38.993177 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:38.993185 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:38.993208 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:45:38.993269 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:38.993275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:38.993523 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:38.993737 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:38.994993 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:38.995990 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:38.997333 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:38.997913 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:38.998055 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:39.000628 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:39.000636 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:39.000651 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:39.000691 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:39.003181 (+ 2490us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.79.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0619 06:45:39.003564 (+ 383us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45780: Request ticket server kudu/127.16.79.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":266,"threads_started":1}
W20260619 06:45:39.003957 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 25652: Not authorized: Client connection negotiation failed: client connection to 127.16.79.66:36843: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.79.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260619 06:45:39.004029 25963 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45780: Request ticket server kudu/127.16.79.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260619 06:45:39.004122 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 25652
I20260619 06:45:39.020571 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 25785
I20260619 06:45:39.033023 25964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:39.021807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50036 (local address 127.16.79.67:36599)
0619 06:45:39.022400 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:39.022407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:39.022432 (+ 25us) server_negotiation.cc:407] Connection header received
0619 06:45:39.022526 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:39.022532 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:39.022758 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:39.022971 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:39.024407 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:39.025438 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:39.026643 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:39.027198 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:39.027344 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:39.029878 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:39.029886 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:39.029900 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:39.029936 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:39.032323 (+ 2387us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.79.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0619 06:45:39.032694 (+ 371us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50036: Request ticket server kudu/127.16.79.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":308,"threads_started":1}
W20260619 06:45:39.033052 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 25785: Not authorized: Client connection negotiation failed: client connection to 127.16.79.67:36599: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.16.79.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260619 06:45:39.033142 25964 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50036: Request ticket server kudu/127.16.79.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260619 06:45:39.033232 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 25785
I20260619 06:45:39.049575 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 25432
I20260619 06:45:39.050967 25494 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:39.166117 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 25432
2026-06-19T06:45:39Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21272 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T06:45:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:45:39Z Disabled control of system clock
I20260619 06:45:39.241792 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:45433
--webserver_interface=127.16.79.126
--webserver_port=0
--builtin_ntp_servers=127.16.79.84:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:45433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 06:45:39.584039 25969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:39.584443 25969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:39.584615 25969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:39.591955 25969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:45:39.592073 25969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:39.592191 25969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:39.592291 25969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:45:39.592365 25969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:45:39.601809 25969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:45433
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:45433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:39.603160 25969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:39.605052 25969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:39.617151 25975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:39.617117 25977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:39.617192 25974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:39.617551 25969 server_base.cc:1034] running on GCE node
I20260619 06:45:39.618522 25969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:39.620016 25969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:39.621223 25969 hybrid_clock.cc:648] HybridClock initialized: now 1781851539621185 us; error 39 us; skew 500 ppm
I20260619 06:45:39.621629 25969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:45:39.623852 25969 webserver.cc:460] Webserver started at http://127.16.79.126:39569/ using document root <none> and password file <none>
I20260619 06:45:39.624636 25969 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:39.624748 25969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:39.625092 25969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:39.628064 25969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "1430a4bd29454d338f1e88e2975a17a6"
format_stamp: "Formatted at 2026-06-19 06:45:39 on dist-test-slave-5w2x"
server_key: "7304370e796c60cb4a1698e4663ba71f"
server_key_iv: "7d59eaa6df11f0c9a1b6b95f84fa8e26"
server_key_version: "encryptionkey@0"
I20260619 06:45:39.628966 25969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "1430a4bd29454d338f1e88e2975a17a6"
format_stamp: "Formatted at 2026-06-19 06:45:39 on dist-test-slave-5w2x"
server_key: "7304370e796c60cb4a1698e4663ba71f"
server_key_iv: "7d59eaa6df11f0c9a1b6b95f84fa8e26"
server_key_version: "encryptionkey@0"
I20260619 06:45:39.634315 25969 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260619 06:45:39.637838 25983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:39.639178 25969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:45:39.639375 25969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "1430a4bd29454d338f1e88e2975a17a6"
format_stamp: "Formatted at 2026-06-19 06:45:39 on dist-test-slave-5w2x"
server_key: "7304370e796c60cb4a1698e4663ba71f"
server_key_iv: "7d59eaa6df11f0c9a1b6b95f84fa8e26"
server_key_version: "encryptionkey@0"
I20260619 06:45:39.639614 25969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:39.661877 25969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:39.662782 25969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:39.663084 25969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:39.682058 25969 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:45433
I20260619 06:45:39.682097 26034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:45433 every 8 connection(s)
I20260619 06:45:39.683584 25969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:45:39.687515 26035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:39.693712 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 25969
I20260619 06:45:39.693924 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:45:39.694566 16701 external_mini_cluster.cc:1428] Setting key 592e1d2453464ae1603cb2ce4c118d35
I20260619 06:45:39.699949 26035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6: Bootstrap starting.
I20260619 06:45:39.704099 26035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:39.705540 26035 log.cc:824] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:39.708843 26035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6: No bootstrap required, opened a new log
I20260619 06:45:39.710608 26038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:39.696905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46240 (local address 127.16.79.126:45433)
0619 06:45:39.698021 (+ 1116us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:39.698045 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:39.698094 (+ 49us) server_negotiation.cc:407] Connection header received
0619 06:45:39.700406 (+ 2312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:39.700436 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:39.701054 (+ 618us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:39.701495 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:39.702102 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:39.704037 (+ 1935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:39.705511 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:39.706598 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:39.706847 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:39.708052 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:39.708060 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:39.708076 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:45:39.708726 (+ 650us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:39.708817 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:45:39.708843 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:39.708979 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:39.709224 (+ 245us) server_negotiation.cc:299] Negotiation successful
0619 06:45:39.709601 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":333,"threads_started":1}
I20260619 06:45:39.717608 26035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1430a4bd29454d338f1e88e2975a17a6" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45433 } }
I20260619 06:45:39.718017 26035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:39.718127 26035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1430a4bd29454d338f1e88e2975a17a6, State: Initialized, Role: FOLLOWER
I20260619 06:45:39.718745 26035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1430a4bd29454d338f1e88e2975a17a6" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45433 } }
I20260619 06:45:39.719036 26035 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:45:39.719185 26035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:45:39.719400 26035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:39.721707 26035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1430a4bd29454d338f1e88e2975a17a6" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45433 } }
I20260619 06:45:39.722223 26035 leader_election.cc:302] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1430a4bd29454d338f1e88e2975a17a6; no voters:
I20260619 06:45:39.722834 26035 leader_election.cc:288] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:45:39.723062 26040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:45:39.723783 26040 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [term 1 LEADER]: Becoming Leader. State: Replica: 1430a4bd29454d338f1e88e2975a17a6, State: Running, Role: LEADER
I20260619 06:45:39.724435 26040 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1430a4bd29454d338f1e88e2975a17a6" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45433 } }
I20260619 06:45:39.724978 26035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:45:39.727471 26042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1430a4bd29454d338f1e88e2975a17a6. Latest consensus state: current_term: 1 leader_uuid: "1430a4bd29454d338f1e88e2975a17a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1430a4bd29454d338f1e88e2975a17a6" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45433 } } }
I20260619 06:45:39.727895 26042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:39.728407 26041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1430a4bd29454d338f1e88e2975a17a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1430a4bd29454d338f1e88e2975a17a6" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 45433 } } }
I20260619 06:45:39.728771 26041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6 [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:39.728976 26049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:45:39.734195 26049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:45:39.743165 26049 catalog_manager.cc:1347] Generated new cluster ID: d0ab7734a4a74cc3bb6fce0729341f09
I20260619 06:45:39.743292 26049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:45:39.771147 26049 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:45:39.772646 26049 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:45:39.782966 26049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1430a4bd29454d338f1e88e2975a17a6: Generated new TSK 0
I20260619 06:45:39.783877 26049 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:45:39.799260 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:45433
--builtin_ntp_servers=127.16.79.84:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 06:45:40.155694 26059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:40.155972 26059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:40.156111 26059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:40.163311 26059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:40.163466 26059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:40.163635 26059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:45:40.173101 26059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:45433
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:40.174382 26059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:40.176224 26059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:40.189581 26065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:40.189636 26064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:40.189658 26067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:40.190476 26059 server_base.cc:1034] running on GCE node
I20260619 06:45:40.191123 26059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:40.191992 26059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:40.193204 26059 hybrid_clock.cc:648] HybridClock initialized: now 1781851540193147 us; error 40 us; skew 500 ppm
I20260619 06:45:40.193538 26059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:45:40.200318 26059 webserver.cc:460] Webserver started at http://127.16.79.65:34981/ using document root <none> and password file <none>
I20260619 06:45:40.201066 26059 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:40.201157 26059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:40.201472 26059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:40.204325 26059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "1af0abe43e474cc798ad8f60a08499df"
format_stamp: "Formatted at 2026-06-19 06:45:40 on dist-test-slave-5w2x"
server_key: "d5380432f0cf5e45d0819e51429ad8ed"
server_key_iv: "0e0b8a7a30bf0a4bf5031c3604ede978"
server_key_version: "encryptionkey@0"
I20260619 06:45:40.205140 26059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "1af0abe43e474cc798ad8f60a08499df"
format_stamp: "Formatted at 2026-06-19 06:45:40 on dist-test-slave-5w2x"
server_key: "d5380432f0cf5e45d0819e51429ad8ed"
server_key_iv: "0e0b8a7a30bf0a4bf5031c3604ede978"
server_key_version: "encryptionkey@0"
I20260619 06:45:40.210338 26059 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:45:40.213730 26073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:40.215031 26059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:45:40.215162 26059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "1af0abe43e474cc798ad8f60a08499df"
format_stamp: "Formatted at 2026-06-19 06:45:40 on dist-test-slave-5w2x"
server_key: "d5380432f0cf5e45d0819e51429ad8ed"
server_key_iv: "0e0b8a7a30bf0a4bf5031c3604ede978"
server_key_version: "encryptionkey@0"
I20260619 06:45:40.215334 26059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:40.235311 26059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:40.236239 26059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:40.236516 26059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:40.237399 26059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:40.238790 26059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:40.238878 26059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:40.238967 26059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:40.239022 26059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:40.272414 26059 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:35033
I20260619 06:45:40.272439 26185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:35033 every 8 connection(s)
I20260619 06:45:40.273775 26059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:45:40.278128 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 26059
I20260619 06:45:40.278324 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:45:40.279176 16701 external_mini_cluster.cc:1428] Setting key ff122e18dae5746ffaabb47b68b0f2c7
I20260619 06:45:40.282354 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:45433
--builtin_ntp_servers=127.16.79.84:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 06:45:40.288651 26188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:40.276300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:53531 (local address 127.16.79.126:45433)
0619 06:45:40.276740 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:40.276745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:40.278039 (+ 1294us) server_negotiation.cc:407] Connection header received
0619 06:45:40.280491 (+ 2452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:40.280498 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:40.280589 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:40.280722 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:40.282095 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:40.282985 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:40.285628 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:40.286046 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:40.286149 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:40.287842 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:40.287846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:40.287850 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:45:40.287868 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:40.287907 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:45:40.287918 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:40.287995 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:40.288297 (+ 302us) server_negotiation.cc:299] Negotiation successful
0619 06:45:40.288389 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":234,"threads_started":1}
I20260619 06:45:40.290580 26189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:40.276818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:45433 (local address 127.16.79.65:53531)
0619 06:45:40.277799 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:40.277857 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:40.280181 (+ 2324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:40.280971 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:40.280981 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:40.281082 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:40.281929 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:40.281956 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:40.283161 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:40.283170 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:40.285462 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:40.285488 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:40.286287 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:40.286296 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:40.286644 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:40.287443 (+ 799us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:45:40.287449 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:40.287517 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:45:40.287635 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:40.288024 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:40.288037 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:40.288049 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 06:45:40.288271 (+ 222us) client_negotiation.cc:240] Negotiation successful
0619 06:45:40.288744 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":265,"threads_started":1}
I20260619 06:45:40.292232 26186 heartbeater.cc:347] Connected to a master server at 127.16.79.126:45433
I20260619 06:45:40.292707 26186 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:40.293547 26186 heartbeater.cc:511] Master 127.16.79.126:45433 requested a full tablet report, sending...
I20260619 06:45:40.295758 26000 ts_manager.cc:194] Registered new tserver with Master: 1af0abe43e474cc798ad8f60a08499df (127.16.79.65:35033)
I20260619 06:45:40.297420 26000 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.79.65:53531
W20260619 06:45:40.617859 26190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:40.618204 26190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:40.618372 26190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:40.626209 26190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:40.626356 26190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:40.626536 26190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:45:40.636097 26190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:45433
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:40.637499 26190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:40.639292 26190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:40.652624 26195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:40.652561 26196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:40.652874 26198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:40.653416 26190 server_base.cc:1034] running on GCE node
I20260619 06:45:40.654062 26190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:40.655087 26190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:40.656299 26190 hybrid_clock.cc:648] HybridClock initialized: now 1781851540656248 us; error 47 us; skew 500 ppm
I20260619 06:45:40.656639 26190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:45:40.658927 26190 webserver.cc:460] Webserver started at http://127.16.79.66:38271/ using document root <none> and password file <none>
I20260619 06:45:40.659646 26190 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:40.659777 26190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:40.660100 26190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:40.662933 26190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "7fdcf472ff254db0b317ebed70dbd411"
format_stamp: "Formatted at 2026-06-19 06:45:40 on dist-test-slave-5w2x"
server_key: "508bf25444cf7f288d71e4e539514ec5"
server_key_iv: "70c0ace12478ce16892fbd84968e1fa0"
server_key_version: "encryptionkey@0"
I20260619 06:45:40.663738 26190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "7fdcf472ff254db0b317ebed70dbd411"
format_stamp: "Formatted at 2026-06-19 06:45:40 on dist-test-slave-5w2x"
server_key: "508bf25444cf7f288d71e4e539514ec5"
server_key_iv: "70c0ace12478ce16892fbd84968e1fa0"
server_key_version: "encryptionkey@0"
I20260619 06:45:40.669523 26190 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.003s
I20260619 06:45:40.673115 26204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:40.674430 26190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:45:40.674615 26190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "7fdcf472ff254db0b317ebed70dbd411"
format_stamp: "Formatted at 2026-06-19 06:45:40 on dist-test-slave-5w2x"
server_key: "508bf25444cf7f288d71e4e539514ec5"
server_key_iv: "70c0ace12478ce16892fbd84968e1fa0"
server_key_version: "encryptionkey@0"
I20260619 06:45:40.674837 26190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:40.691587 26190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:40.692526 26190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:40.692827 26190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:40.693856 26190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:40.695334 26190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:40.695470 26190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:40.695601 26190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:40.695689 26190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:40.729121 26190 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:45589
I20260619 06:45:40.729168 26316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:45589 every 8 connection(s)
I20260619 06:45:40.730412 26190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:45:40.737304 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 26190
I20260619 06:45:40.737486 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:45:40.738091 16701 external_mini_cluster.cc:1428] Setting key 7aa1d87e6ee55502a75bcecf137b64ef
I20260619 06:45:40.740823 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:45433
--builtin_ntp_servers=127.16.79.84:45355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 06:45:40.744208 26188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:40.733021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:44267 (local address 127.16.79.126:45433)
0619 06:45:40.733191 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:40.733196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:40.734678 (+ 1482us) server_negotiation.cc:407] Connection header received
0619 06:45:40.736417 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:40.736421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:40.736486 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:40.736574 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:40.737712 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:40.738737 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:40.740963 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:40.741731 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:40.741877 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:40.743197 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:40.743205 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:40.743213 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:45:40.743237 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:40.743278 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:45:40.743295 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:40.743387 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:40.743657 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 06:45:40.743765 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:45:40.745235 26319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:40.733600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:45433 (local address 127.16.79.66:44267)
0619 06:45:40.734506 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:40.734548 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:40.736093 (+ 1545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:40.736783 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:40.736790 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:40.736873 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:40.737505 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:40.737531 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:40.739130 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:40.739135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:40.740810 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:40.740826 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:40.741960 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:40.741965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:40.742167 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:40.742886 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:45:40.742889 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:40.742931 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:45:40.743011 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:40.743394 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:40.743401 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:40.743407 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 06:45:40.743616 (+ 209us) client_negotiation.cc:240] Negotiation successful
0619 06:45:40.743985 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":271,"threads_started":1}
I20260619 06:45:40.746625 26317 heartbeater.cc:347] Connected to a master server at 127.16.79.126:45433
I20260619 06:45:40.746946 26317 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:40.747839 26317 heartbeater.cc:511] Master 127.16.79.126:45433 requested a full tablet report, sending...
I20260619 06:45:40.749358 26000 ts_manager.cc:194] Registered new tserver with Master: 7fdcf472ff254db0b317ebed70dbd411 (127.16.79.66:45589)
I20260619 06:45:40.750578 26000 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.79.66:44267
W20260619 06:45:41.069376 26320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:41.069664 26320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:41.069751 26320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:41.077018 26320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:41.077131 26320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:41.077243 26320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:45:41.086501 26320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:45355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:45433
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:41.087733 26320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:41.089427 26320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:41.102006 26328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:41.101962 26325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:41.101998 26326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:41.102794 26320 server_base.cc:1034] running on GCE node
I20260619 06:45:41.103405 26320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:41.104378 26320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:41.105607 26320 hybrid_clock.cc:648] HybridClock initialized: now 1781851541105552 us; error 35 us; skew 500 ppm
I20260619 06:45:41.105913 26320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:45:41.108280 26320 webserver.cc:460] Webserver started at http://127.16.79.67:34551/ using document root <none> and password file <none>
I20260619 06:45:41.109040 26320 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:41.109160 26320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:41.109503 26320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:41.112488 26320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "f9337ff22f01466c8c81027226672d66"
format_stamp: "Formatted at 2026-06-19 06:45:41 on dist-test-slave-5w2x"
server_key: "b3182ad510c0c3e9f7e6c02d7b016596"
server_key_iv: "f99f0e4f2cc14752d317500a267aba11"
server_key_version: "encryptionkey@0"
I20260619 06:45:41.113299 26320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "f9337ff22f01466c8c81027226672d66"
format_stamp: "Formatted at 2026-06-19 06:45:41 on dist-test-slave-5w2x"
server_key: "b3182ad510c0c3e9f7e6c02d7b016596"
server_key_iv: "f99f0e4f2cc14752d317500a267aba11"
server_key_version: "encryptionkey@0"
I20260619 06:45:41.118593 26320 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260619 06:45:41.122272 26334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:41.123729 26320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:45:41.123931 26320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "f9337ff22f01466c8c81027226672d66"
format_stamp: "Formatted at 2026-06-19 06:45:41 on dist-test-slave-5w2x"
server_key: "b3182ad510c0c3e9f7e6c02d7b016596"
server_key_iv: "f99f0e4f2cc14752d317500a267aba11"
server_key_version: "encryptionkey@0"
I20260619 06:45:41.124224 26320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:41.140841 26320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:41.141821 26320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:41.142161 26320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:41.143208 26320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:41.144786 26320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:41.144907 26320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:41.145051 26320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:41.145138 26320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:41.178741 26320 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:45481
I20260619 06:45:41.178803 26446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:45481 every 8 connection(s)
I20260619 06:45:41.180128 26320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:45:41.184882 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 26320
I20260619 06:45:41.185053 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:45:41.185636 16701 external_mini_cluster.cc:1428] Setting key 993200ff3aeae9c3ddccea07512b4fbc
I20260619 06:45:41.192986 26188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:41.182735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:57395 (local address 127.16.79.126:45433)
0619 06:45:41.182902 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:41.182907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:41.184546 (+ 1639us) server_negotiation.cc:407] Connection header received
0619 06:45:41.186820 (+ 2274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:41.186825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:41.186913 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:41.187020 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:41.188061 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:41.188837 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:41.190363 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:41.190770 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:41.190902 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:41.192180 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:41.192188 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:41.192193 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:45:41.192214 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:41.192253 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:45:41.192265 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:41.192356 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:41.192577 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 06:45:41.192667 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:45:41.194011 26449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:41.183292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:45433 (local address 127.16.79.67:57395)
0619 06:45:41.184357 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:41.184394 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:41.186548 (+ 2154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:41.187256 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:41.187264 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:41.187353 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:41.187894 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:41.187910 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:41.188995 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:41.189000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:41.190220 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:41.190231 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:41.190994 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:41.191000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:41.191224 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:41.191841 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:45:41.191843 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:41.191880 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:45:41.191968 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:41.192366 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:41.192378 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:41.192392 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 06:45:41.192569 (+ 177us) client_negotiation.cc:240] Negotiation successful
0619 06:45:41.192945 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":318,"threads_started":1}
I20260619 06:45:41.195292 26447 heartbeater.cc:347] Connected to a master server at 127.16.79.126:45433
I20260619 06:45:41.195577 26447 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:41.196391 26447 heartbeater.cc:511] Master 127.16.79.126:45433 requested a full tablet report, sending...
I20260619 06:45:41.197729 26000 ts_manager.cc:194] Registered new tserver with Master: f9337ff22f01466c8c81027226672d66 (127.16.79.67:45481)
I20260619 06:45:41.198494 26000 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.79.67:57395
I20260619 06:45:41.200671 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 06:45:41.206651 26188 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:41.205021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46248 (local address 127.16.79.126:45433)
0619 06:45:41.205248 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:41.205256 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:41.205750 (+ 494us) server_negotiation.cc:407] Connection header received
0619 06:45:41.205882 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:41.205885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:41.205958 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:41.206041 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:41.206424 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46248: BlockingRecv error: recv got EOF from 127.0.0.1:46248 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 06:45:41.207224 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 26059
I20260619 06:45:41.215055 26456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:41.208221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58116 (local address 127.16.79.65:35033)
0619 06:45:41.208752 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:41.208758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:41.208782 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:45:41.208838 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:41.208844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:41.209044 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:41.209207 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:41.209591 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:41.210533 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:41.211799 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:41.212413 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:41.212552 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:41.213574 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:41.213582 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:41.213595 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:45:41.214278 (+ 683us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:41.214333 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:45:41.214352 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:41.214481 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:41.214593 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 06:45:41.214732 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":244,"threads_started":1}
I20260619 06:45:41.216318 26181 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:41.299399 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 26059
I20260619 06:45:41.315470 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 26190
I20260619 06:45:41.323490 26458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:41.316506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57858 (local address 127.16.79.66:45589)
0619 06:45:41.317092 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:41.317099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:41.317127 (+ 28us) server_negotiation.cc:407] Connection header received
0619 06:45:41.317187 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:41.317193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:41.317412 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:41.317551 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:41.317971 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:41.319028 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:41.320425 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:41.321010 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:41.321114 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:41.322112 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:41.322121 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:41.322135 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:45:41.322742 (+ 607us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:41.322787 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:45:41.322806 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:41.322894 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:41.323051 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 06:45:41.323172 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":284,"threads_started":1}
I20260619 06:45:41.324820 26312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:41.406927 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 26190
I20260619 06:45:41.423477 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 26320
I20260619 06:45:41.432581 26460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:41.424420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.16.79.67:45481)
0619 06:45:41.425074 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:41.425081 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:41.425111 (+ 30us) server_negotiation.cc:407] Connection header received
0619 06:45:41.425168 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:41.425174 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:41.425445 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:41.425615 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:41.426145 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:41.427320 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:41.429095 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:41.429757 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:41.429897 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:41.431046 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:41.431055 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:41.431068 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:45:41.431743 (+ 675us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:41.431789 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:45:41.431807 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:41.431939 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:41.432062 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 06:45:41.432210 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":323,"threads_started":1}
I20260619 06:45:41.434033 26442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:41.517894 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 26320
I20260619 06:45:41.533653 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 25969
I20260619 06:45:41.534811 26030 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:41.636201 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 25969
2026-06-19T06:45:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2464 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T06:45:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:45:41Z Disabled control of system clock
I20260619 06:45:41.706616 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:43975
--webserver_interface=127.16.79.126
--webserver_port=0
--builtin_ntp_servers=127.16.79.84:42521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:43975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 06:45:42.032838 26466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:42.033121 26466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:42.033202 26466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:42.040359 26466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:45:42.040481 26466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:42.040519 26466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:42.040580 26466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:45:42.040624 26466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:45:42.049796 26466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:42521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:43975
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:43975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:42.051017 26466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:42.052718 26466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:42.064098 26471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:42.064093 26472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:42.064127 26474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:42.065315 26466 server_base.cc:1034] running on GCE node
I20260619 06:45:42.066047 26466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:42.067435 26466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:42.068636 26466 hybrid_clock.cc:648] HybridClock initialized: now 1781851542068587 us; error 51 us; skew 500 ppm
I20260619 06:45:42.068971 26466 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:45:42.071290 26466 webserver.cc:460] Webserver started at http://127.16.79.126:42577/ using document root <none> and password file <none>
I20260619 06:45:42.071981 26466 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:42.072094 26466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:42.072444 26466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:42.076742 26466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "6e32419c29324e2ba417b4c897e75037"
format_stamp: "Formatted at 2026-06-19 06:45:42 on dist-test-slave-5w2x"
server_key: "8c0fa796478bbe317a3a5589c61ae8ca"
server_key_iv: "16ae395a015128b6cb64da002a009cc4"
server_key_version: "encryptionkey@0"
I20260619 06:45:42.077529 26466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "6e32419c29324e2ba417b4c897e75037"
format_stamp: "Formatted at 2026-06-19 06:45:42 on dist-test-slave-5w2x"
server_key: "8c0fa796478bbe317a3a5589c61ae8ca"
server_key_iv: "16ae395a015128b6cb64da002a009cc4"
server_key_version: "encryptionkey@0"
I20260619 06:45:42.082353 26466 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260619 06:45:42.085626 26480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:42.087059 26466 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:45:42.087188 26466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "6e32419c29324e2ba417b4c897e75037"
format_stamp: "Formatted at 2026-06-19 06:45:42 on dist-test-slave-5w2x"
server_key: "8c0fa796478bbe317a3a5589c61ae8ca"
server_key_iv: "16ae395a015128b6cb64da002a009cc4"
server_key_version: "encryptionkey@0"
I20260619 06:45:42.087437 26466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:42.093129 26466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:42.093808 26466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:42.094044 26466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:42.111932 26466 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:43975
I20260619 06:45:42.111974 26531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:43975 every 8 connection(s)
I20260619 06:45:42.113300 26466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:45:42.117013 26532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:42.118213 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 26466
I20260619 06:45:42.118428 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:45:42.119043 16701 external_mini_cluster.cc:1428] Setting key a6258dbc6da1941b50107fa3ec30c2e0
I20260619 06:45:42.128230 26535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:42.121114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48276 (local address 127.16.79.126:43975)
0619 06:45:42.122250 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:42.122265 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:42.122314 (+ 49us) server_negotiation.cc:407] Connection header received
0619 06:45:42.124031 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:42.124063 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:42.124695 (+ 632us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:42.125181 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:42.126174 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:42.126184 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:42.126202 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:45:42.126219 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:42.126288 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:45:42.126309 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:42.126525 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:42.126671 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 06:45:42.126939 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":347,"threads_started":1}
I20260619 06:45:42.131435 26532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037: Bootstrap starting.
I20260619 06:45:42.134419 26532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:42.135373 26532 log.cc:824] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:42.138270 26532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037: No bootstrap required, opened a new log
I20260619 06:45:42.145674 26532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e32419c29324e2ba417b4c897e75037" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43975 } }
I20260619 06:45:42.146018 26532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:42.146092 26532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e32419c29324e2ba417b4c897e75037, State: Initialized, Role: FOLLOWER
I20260619 06:45:42.146607 26532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e32419c29324e2ba417b4c897e75037" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43975 } }
I20260619 06:45:42.146795 26532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:45:42.146919 26532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:45:42.147060 26532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:42.149508 26532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e32419c29324e2ba417b4c897e75037" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43975 } }
I20260619 06:45:42.149950 26532 leader_election.cc:302] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e32419c29324e2ba417b4c897e75037; no voters:
I20260619 06:45:42.150478 26532 leader_election.cc:288] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:45:42.150633 26537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:45:42.151465 26537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [term 1 LEADER]: Becoming Leader. State: Replica: 6e32419c29324e2ba417b4c897e75037, State: Running, Role: LEADER
I20260619 06:45:42.152006 26537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e32419c29324e2ba417b4c897e75037" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43975 } }
I20260619 06:45:42.152649 26532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:45:42.154840 26538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e32419c29324e2ba417b4c897e75037" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e32419c29324e2ba417b4c897e75037" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43975 } } }
I20260619 06:45:42.155238 26538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:42.155025 26539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e32419c29324e2ba417b4c897e75037. Latest consensus state: current_term: 1 leader_uuid: "6e32419c29324e2ba417b4c897e75037" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e32419c29324e2ba417b4c897e75037" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43975 } } }
I20260619 06:45:42.155544 26539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037 [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:42.156311 26544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:45:42.163609 26544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:45:42.173579 26544 catalog_manager.cc:1347] Generated new cluster ID: 36003d23749d46eb848577f6ffc2af5d
I20260619 06:45:42.173712 26544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:45:42.190045 26544 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:45:42.190572 26544 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:45:42.208066 26544 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e32419c29324e2ba417b4c897e75037: Generated new TSK 0
I20260619 06:45:42.208994 26544 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:45:42.226517 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43975
--builtin_ntp_servers=127.16.79.84:42521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 06:45:42.574611 26556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:42.574878 26556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:42.575038 26556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:42.582368 26556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:42.582469 26556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:42.582607 26556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:45:42.591952 26556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:42521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43975
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:42.593273 26556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:42.595041 26556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:42.608537 26561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:42.608588 26562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:42.608773 26564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:42.609689 26556 server_base.cc:1034] running on GCE node
I20260619 06:45:42.610306 26556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:42.611299 26556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:42.612519 26556 hybrid_clock.cc:648] HybridClock initialized: now 1781851542612470 us; error 69 us; skew 500 ppm
I20260619 06:45:42.612849 26556 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:45:42.615132 26556 webserver.cc:460] Webserver started at http://127.16.79.65:34739/ using document root <none> and password file <none>
I20260619 06:45:42.615836 26556 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:42.615969 26556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:42.616356 26556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:42.619148 26556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "06205c626c3c4711a052a124789ee99b"
format_stamp: "Formatted at 2026-06-19 06:45:42 on dist-test-slave-5w2x"
server_key: "19dc64f38f471a69bcd13189813daabd"
server_key_iv: "2941864cd763119cacc0ab84f59ee760"
server_key_version: "encryptionkey@0"
I20260619 06:45:42.619969 26556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "06205c626c3c4711a052a124789ee99b"
format_stamp: "Formatted at 2026-06-19 06:45:42 on dist-test-slave-5w2x"
server_key: "19dc64f38f471a69bcd13189813daabd"
server_key_iv: "2941864cd763119cacc0ab84f59ee760"
server_key_version: "encryptionkey@0"
I20260619 06:45:42.625219 26556 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260619 06:45:42.628793 26570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:42.630193 26556 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:45:42.630412 26556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "06205c626c3c4711a052a124789ee99b"
format_stamp: "Formatted at 2026-06-19 06:45:42 on dist-test-slave-5w2x"
server_key: "19dc64f38f471a69bcd13189813daabd"
server_key_iv: "2941864cd763119cacc0ab84f59ee760"
server_key_version: "encryptionkey@0"
I20260619 06:45:42.630656 26556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:42.636319 26556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:42.637125 26556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:42.637395 26556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:42.638275 26556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:42.639647 26556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:42.639766 26556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:42.639901 26556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:42.639988 26556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:42.672844 26556 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:33137
I20260619 06:45:42.672899 26682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:33137 every 8 connection(s)
I20260619 06:45:42.674181 26556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:45:42.682236 26685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:42.676777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:59489 (local address 127.16.79.126:43975)
0619 06:45:42.677216 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:42.677222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:42.678480 (+ 1258us) server_negotiation.cc:407] Connection header received
0619 06:45:42.680067 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:42.680072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:42.680120 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:42.680267 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:42.681462 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:42.681467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:42.681471 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:45:42.681485 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:42.681530 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:45:42.681542 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:42.681641 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:42.681793 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 06:45:42.681893 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":208,"threads_started":1}
I20260619 06:45:42.682320 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 26556
I20260619 06:45:42.682538 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:45:42.683104 26686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:42.677421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:43975 (local address 127.16.79.65:59489)
0619 06:45:42.678313 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:42.678352 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:42.679823 (+ 1471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:42.680478 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:42.680486 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:42.680582 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:42.681180 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:45:42.681182 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:42.681220 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:45:42.681305 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:42.681626 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:42.681633 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:42.681638 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 06:45:42.681842 (+ 204us) client_negotiation.cc:240] Negotiation successful
0619 06:45:42.682042 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":257,"threads_started":1}
I20260619 06:45:42.683148 16701 external_mini_cluster.cc:1428] Setting key 33f64ed9a56d304396fb1ba3ab178097
I20260619 06:45:42.684772 26683 heartbeater.cc:347] Connected to a master server at 127.16.79.126:43975
I20260619 06:45:42.685091 26683 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:42.685674 26683 heartbeater.cc:511] Master 127.16.79.126:43975 requested a full tablet report, sending...
I20260619 06:45:42.686062 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43975
--builtin_ntp_servers=127.16.79.84:42521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 06:45:42.687821 26497 ts_manager.cc:194] Registered new tserver with Master: 06205c626c3c4711a052a124789ee99b (127.16.79.65:33137)
W20260619 06:45:43.010447 26687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:43.010746 26687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:43.010843 26687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:43.018172 26687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:43.018275 26687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:43.018385 26687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:45:43.027523 26687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:42521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43975
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:43.028793 26687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:43.030455 26687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:43.043517 26692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:43.043470 26693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:43.043551 26695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:43.044538 26687 server_base.cc:1034] running on GCE node
I20260619 06:45:43.045183 26687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:43.046197 26687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:43.047458 26687 hybrid_clock.cc:648] HybridClock initialized: now 1781851543047412 us; error 38 us; skew 500 ppm
I20260619 06:45:43.047806 26687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:45:43.050140 26687 webserver.cc:460] Webserver started at http://127.16.79.66:36691/ using document root <none> and password file <none>
I20260619 06:45:43.050853 26687 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:43.050974 26687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:43.051307 26687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:43.054349 26687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "70744b67c9e04ba0acef82a0bb0e287d"
format_stamp: "Formatted at 2026-06-19 06:45:43 on dist-test-slave-5w2x"
server_key: "017cedde8489013df6ff3efe08d58a6f"
server_key_iv: "e2d7a74ecd49ba6f8418a841d43d3871"
server_key_version: "encryptionkey@0"
I20260619 06:45:43.055116 26687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "70744b67c9e04ba0acef82a0bb0e287d"
format_stamp: "Formatted at 2026-06-19 06:45:43 on dist-test-slave-5w2x"
server_key: "017cedde8489013df6ff3efe08d58a6f"
server_key_iv: "e2d7a74ecd49ba6f8418a841d43d3871"
server_key_version: "encryptionkey@0"
I20260619 06:45:43.060238 26687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 06:45:43.063797 26701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:43.065243 26687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:45:43.065454 26687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "70744b67c9e04ba0acef82a0bb0e287d"
format_stamp: "Formatted at 2026-06-19 06:45:43 on dist-test-slave-5w2x"
server_key: "017cedde8489013df6ff3efe08d58a6f"
server_key_iv: "e2d7a74ecd49ba6f8418a841d43d3871"
server_key_version: "encryptionkey@0"
I20260619 06:45:43.065681 26687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:43.071470 26687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:43.072422 26687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:43.072690 26687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:43.073571 26687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:43.074925 26687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:43.075038 26687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:43.075170 26687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:43.075264 26687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:43.107528 26687 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:42591
I20260619 06:45:43.107558 26813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:42591 every 8 connection(s)
I20260619 06:45:43.108913 26687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:45:43.118038 26685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:43.111464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:51641 (local address 127.16.79.126:43975)
0619 06:45:43.111655 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:43.111660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:43.113269 (+ 1609us) server_negotiation.cc:407] Connection header received
0619 06:45:43.115589 (+ 2320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:43.115593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:43.115649 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:43.115740 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:43.117242 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:43.117250 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:43.117253 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:45:43.117269 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:43.117312 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:45:43.117324 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:43.117414 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:43.117632 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 06:45:43.117709 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 06:45:43.118076 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 26687
I20260619 06:45:43.118245 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:45:43.118726 16701 external_mini_cluster.cc:1428] Setting key 2b56c7f4aea32b17dcd514d422ffa045
I20260619 06:45:43.119131 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:43.111987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:43975 (local address 127.16.79.66:51641)
0619 06:45:43.113093 (+ 1106us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:43.113144 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:43.115288 (+ 2144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:43.116013 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:43.116025 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:43.116177 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:43.116863 (+ 686us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:45:43.116867 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:43.116923 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:45:43.117035 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:43.117423 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:43.117432 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:43.117440 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 06:45:43.117733 (+ 293us) client_negotiation.cc:240] Negotiation successful
0619 06:45:43.118018 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":330,"threads_started":1}
I20260619 06:45:43.120640 26814 heartbeater.cc:347] Connected to a master server at 127.16.79.126:43975
I20260619 06:45:43.120992 26814 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:43.121702 26814 heartbeater.cc:511] Master 127.16.79.126:43975 requested a full tablet report, sending...
I20260619 06:45:43.122107 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43975
--builtin_ntp_servers=127.16.79.84:42521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 06:45:43.123364 26497 ts_manager.cc:194] Registered new tserver with Master: 70744b67c9e04ba0acef82a0bb0e287d (127.16.79.66:42591)
W20260619 06:45:43.447722 26817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:43.448060 26817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:43.448225 26817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:43.455653 26817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:43.455791 26817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:43.455971 26817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:45:43.465559 26817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:42521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43975
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:43.466876 26817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:43.468725 26817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:43.482264 26823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:43.482334 26825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:43.482364 26822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:43.483106 26817 server_base.cc:1034] running on GCE node
I20260619 06:45:43.483803 26817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:43.484925 26817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:43.486132 26817 hybrid_clock.cc:648] HybridClock initialized: now 1781851543486088 us; error 35 us; skew 500 ppm
I20260619 06:45:43.486482 26817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:45:43.489128 26817 webserver.cc:460] Webserver started at http://127.16.79.67:36425/ using document root <none> and password file <none>
I20260619 06:45:43.489948 26817 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:43.490233 26817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:43.490635 26817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:43.493760 26817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "6ac7fde8168841fa97baaf6db007df6b"
format_stamp: "Formatted at 2026-06-19 06:45:43 on dist-test-slave-5w2x"
server_key: "672b7af4154a19643c88c32249220530"
server_key_iv: "f8b5efee65100f126c69bd29f252df62"
server_key_version: "encryptionkey@0"
I20260619 06:45:43.494542 26817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "6ac7fde8168841fa97baaf6db007df6b"
format_stamp: "Formatted at 2026-06-19 06:45:43 on dist-test-slave-5w2x"
server_key: "672b7af4154a19643c88c32249220530"
server_key_iv: "f8b5efee65100f126c69bd29f252df62"
server_key_version: "encryptionkey@0"
I20260619 06:45:43.499714 26817 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:45:43.503214 26831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:43.504657 26817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:45:43.504828 26817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "6ac7fde8168841fa97baaf6db007df6b"
format_stamp: "Formatted at 2026-06-19 06:45:43 on dist-test-slave-5w2x"
server_key: "672b7af4154a19643c88c32249220530"
server_key_iv: "f8b5efee65100f126c69bd29f252df62"
server_key_version: "encryptionkey@0"
I20260619 06:45:43.505050 26817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:43.510699 26817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:43.511482 26817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:43.511750 26817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:43.512717 26817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:43.514088 26817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:43.514173 26817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:43.514297 26817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:43.514384 26817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:43.547884 26817 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:34089
I20260619 06:45:43.547930 26943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:34089 every 8 connection(s)
I20260619 06:45:43.549324 26817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:45:43.554450 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 26817
I20260619 06:45:43.554657 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:45:43.555186 16701 external_mini_cluster.cc:1428] Setting key 4d0150de3f60334e16a2e90863082f1a
I20260619 06:45:43.558844 26685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:43.552127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:39501 (local address 127.16.79.126:43975)
0619 06:45:43.552385 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:43.552391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:43.553993 (+ 1602us) server_negotiation.cc:407] Connection header received
0619 06:45:43.556391 (+ 2398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:43.556395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:43.556447 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:43.556574 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:43.558100 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:43.558104 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:43.558107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:45:43.558123 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:43.558178 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:45:43.558194 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:43.558310 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:43.558543 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 06:45:43.558608 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 06:45:43.559825 26946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:43.552782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:43975 (local address 127.16.79.67:39501)
0619 06:45:43.553819 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:43.553856 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:43.556031 (+ 2175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:43.556859 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:43.556869 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:43.556980 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:43.557711 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:45:43.557717 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:43.557779 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:45:43.557899 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:43.558302 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:43.558313 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:43.558321 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 06:45:43.558608 (+ 287us) client_negotiation.cc:240] Negotiation successful
0619 06:45:43.558821 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":298,"threads_started":1}
I20260619 06:45:43.561203 26944 heartbeater.cc:347] Connected to a master server at 127.16.79.126:43975
I20260619 06:45:43.561571 26944 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:43.562248 26944 heartbeater.cc:511] Master 127.16.79.126:43975 requested a full tablet report, sending...
I20260619 06:45:43.563740 26497 ts_manager.cc:194] Registered new tserver with Master: 6ac7fde8168841fa97baaf6db007df6b (127.16.79.67:34089)
I20260619 06:45:43.571002 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 06:45:43.577519 26685 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:45:43.575781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48286 (local address 127.16.79.126:43975)
0619 06:45:43.575940 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:43.575945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:43.576549 (+ 604us) server_negotiation.cc:407] Connection header received
0619 06:45:43.576757 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:43.576769 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:43.576819 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:43.576922 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:43.577262 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48286: BlockingRecv error: recv got EOF from 127.0.0.1:48286 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:45:43.578061 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 26556
I20260619 06:45:43.581532 26953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:43.579029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42838 (local address 127.16.79.65:33137)
0619 06:45:43.579575 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:43.579582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:43.579609 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:45:43.579667 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:43.579674 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:43.579888 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:43.580023 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:43.580927 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:43.580934 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:43.580945 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:45:43.580974 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:43.581017 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:45:43.581032 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:43.581138 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:43.581216 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 06:45:43.581291 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":267,"threads_started":1}
I20260619 06:45:43.582703 26678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:43.664287 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 26556
I20260619 06:45:43.680831 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 26687
I20260619 06:45:43.684489 26955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:43.681813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55264 (local address 127.16.79.66:42591)
0619 06:45:43.682367 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:43.682374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:43.682401 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:45:43.682465 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:43.682472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:43.682707 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:43.682839 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:43.683759 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:43.683765 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:43.683777 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:45:43.683794 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:43.683831 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:45:43.683844 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:43.683949 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:43.684043 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 06:45:43.684180 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":273,"threads_started":1}
I20260619 06:45:43.685876 26809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:43.766541 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 26687
I20260619 06:45:43.782440 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 26817
I20260619 06:45:43.786280 26957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:43.783503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58292 (local address 127.16.79.67:34089)
0619 06:45:43.784227 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:43.784233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:43.784266 (+ 33us) server_negotiation.cc:407] Connection header received
0619 06:45:43.784327 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:43.784333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:43.784560 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:43.784685 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:43.785581 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:43.785587 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:43.785598 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:45:43.785613 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:43.785649 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:45:43.785662 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:43.785789 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:43.785837 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 06:45:43.785943 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":366,"threads_started":1}
I20260619 06:45:43.787549 26939 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:43.867200 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 26817
I20260619 06:45:43.882305 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 26466
I20260619 06:45:43.883541 26527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:43.984082 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 26466
2026-06-19T06:45:43Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2347 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:45:44 dist-test-slave-5w2x krb5kdc[26962](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:45:44 dist-test-slave-5w2x krb5kdc[26962](info): set up 2 sockets
Jun 19 06:45:44 dist-test-slave-5w2x krb5kdc[26962](info): commencing operation
krb5kdc: starting...
W20260619 06:45:45.081187 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.005s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:45:45 dist-test-slave-5w2x krb5kdc[26962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:45:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:45:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:45:45.313999 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:40233
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:40233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:45.640664 26975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:45.640961 26975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:45.641144 26975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:45.648501 26975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:45:45.648639 26975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:45.648739 26975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:45.648834 26975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:45:45.648910 26975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:45:45.658903 26975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:40233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:40233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:45.660337 26975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:45.661614 26975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:45.673153 26983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:45.673247 26981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:45.673475 26980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:45.674364 26975 server_base.cc:1034] running on GCE node
I20260619 06:45:45.675184 26975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:45.676643 26975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:45.677855 26975 hybrid_clock.cc:648] HybridClock initialized: now 1781851545677801 us; error 40 us; skew 500 ppm
I20260619 06:45:45.678220 26975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:45 dist-test-slave-5w2x krb5kdc[26962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851545, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:45.694921 26975 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:45:45.696774 26975 webserver.cc:460] Webserver started at http://127.16.79.126:38151/ using document root <none> and password file <none>
I20260619 06:45:45.697520 26975 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:45.697672 26975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:45.697996 26975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:45.701122 26975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "e048493c59774e229c8cb96a4c97111f"
format_stamp: "Formatted at 2026-06-19 06:45:45 on dist-test-slave-5w2x"
server_key: "77a2146036370f7b84ab92f9971d0bd2"
server_key_iv: "f05551a161c27f474e24b025c11bbdc5"
server_key_version: "encryptionkey@0"
I20260619 06:45:45.701894 26975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "e048493c59774e229c8cb96a4c97111f"
format_stamp: "Formatted at 2026-06-19 06:45:45 on dist-test-slave-5w2x"
server_key: "77a2146036370f7b84ab92f9971d0bd2"
server_key_iv: "f05551a161c27f474e24b025c11bbdc5"
server_key_version: "encryptionkey@0"
I20260619 06:45:45.707257 26975 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:45:45.710983 26990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:45.712527 26975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 06:45:45.712739 26975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "e048493c59774e229c8cb96a4c97111f"
format_stamp: "Formatted at 2026-06-19 06:45:45 on dist-test-slave-5w2x"
server_key: "77a2146036370f7b84ab92f9971d0bd2"
server_key_iv: "f05551a161c27f474e24b025c11bbdc5"
server_key_version: "encryptionkey@0"
I20260619 06:45:45.712965 26975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:45.728956 26975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:45.733148 26975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:45.733458 26975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:45.752223 27041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:40233 every 8 connection(s)
I20260619 06:45:45.752254 26975 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:40233
I20260619 06:45:45.753645 26975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:45:45.755892 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 26975
I20260619 06:45:45.756127 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:45:45.756803 16701 external_mini_cluster.cc:1428] Setting key 5d883e4a1c1d2551ae81b8d3bd3721f8
I20260619 06:45:45.757503 27042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:45.771749 27042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f: Bootstrap starting.
Jun 19 06:45:45 dist-test-slave-5w2x krb5kdc[26962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:45:45.775810 27042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:45.777211 27042 log.cc:824] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:45.780743 27042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f: No bootstrap required, opened a new log
I20260619 06:45:45.781750 27045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:45.758739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44666 (local address 127.16.79.126:40233)
0619 06:45:45.759488 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:45.759502 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:45.759540 (+ 38us) server_negotiation.cc:407] Connection header received
0619 06:45:45.761887 (+ 2347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:45.761908 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:45.762331 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:45.762670 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:45.764550 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:45.765805 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:45.767575 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:45.768348 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:45.768489 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:45.773062 (+ 4573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:45.773073 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:45.773093 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:45.773146 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:45.777087 (+ 3941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:45.777862 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:45.777871 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:45.777882 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:45.778136 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:45.778603 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:45.778609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:45.778612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:45.779280 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:45.779586 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:45.779942 (+ 356us) server_negotiation.cc:299] Negotiation successful
0619 06:45:45.780399 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":358,"threads_started":1}
I20260619 06:45:45.789130 27042 raft_consensus.cc:348] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e048493c59774e229c8cb96a4c97111f" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40233 } }
I20260619 06:45:45.789469 27042 raft_consensus.cc:374] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:45.789542 27042 raft_consensus.cc:729] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e048493c59774e229c8cb96a4c97111f, State: Initialized, Role: FOLLOWER
I20260619 06:45:45.790091 27042 consensus_queue.cc:260] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e048493c59774e229c8cb96a4c97111f" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40233 } }
I20260619 06:45:45.790308 27042 raft_consensus.cc:388] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:45:45.790405 27042 raft_consensus.cc:482] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:45:45.790527 27042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:45.792944 27042 raft_consensus.cc:504] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e048493c59774e229c8cb96a4c97111f" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40233 } }
I20260619 06:45:45.793403 27042 leader_election.cc:302] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e048493c59774e229c8cb96a4c97111f; no voters:
I20260619 06:45:45.793902 27042 leader_election.cc:288] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:45:45.794041 27047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:45:45.794713 27047 raft_consensus.cc:686] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [term 1 LEADER]: Becoming Leader. State: Replica: e048493c59774e229c8cb96a4c97111f, State: Running, Role: LEADER
I20260619 06:45:45.795270 27047 consensus_queue.cc:237] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e048493c59774e229c8cb96a4c97111f" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40233 } }
I20260619 06:45:45.796085 27042 sys_catalog.cc:564] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:45:45.798007 27048 sys_catalog.cc:455] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e048493c59774e229c8cb96a4c97111f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e048493c59774e229c8cb96a4c97111f" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40233 } } }
I20260619 06:45:45.798333 27048 sys_catalog.cc:458] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:45.798780 27049 sys_catalog.cc:455] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e048493c59774e229c8cb96a4c97111f. Latest consensus state: current_term: 1 leader_uuid: "e048493c59774e229c8cb96a4c97111f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e048493c59774e229c8cb96a4c97111f" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40233 } } }
I20260619 06:45:45.799103 27049 sys_catalog.cc:458] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f [sys.catalog]: This master's current role is: LEADER
I20260619 06:45:45.799292 27053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:45:45.804558 27053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:45:45.814410 27053 catalog_manager.cc:1347] Generated new cluster ID: afffef9d5ebf445f8e22ba219f8fc601
I20260619 06:45:45.814553 27053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:45:45.830808 27053 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:45:45.832267 27053 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:45:45.839550 27053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f: Generated new TSK 0
I20260619 06:45:45.840536 27053 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:45:45.904963 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:40233
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:46.241035 27068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:46.241313 27068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:46.241405 27068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:46.248656 27068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:46.248770 27068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:46.248883 27068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:45:46.258653 27068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:40233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:46.259920 27068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:46.261257 27068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:46.273940 27073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:46.273933 27074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:46.273937 27076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:46.274986 27068 server_base.cc:1034] running on GCE node
I20260619 06:45:46.275787 27068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:46.276873 27068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:46.278131 27068 hybrid_clock.cc:648] HybridClock initialized: now 1781851546278075 us; error 66 us; skew 500 ppm
I20260619 06:45:46.278489 27068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:46 dist-test-slave-5w2x krb5kdc[26962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851546, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:46.282054 27068 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:45:46.283627 27068 webserver.cc:460] Webserver started at http://127.16.79.65:42519/ using document root <none> and password file <none>
I20260619 06:45:46.284475 27068 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:46.284628 27068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:46.284958 27068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:46.288015 27068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "e58c9d8c9feb49c5b50595efd57fb1b9"
format_stamp: "Formatted at 2026-06-19 06:45:46 on dist-test-slave-5w2x"
server_key: "d0b41bc63b9b717651147fd1cd591352"
server_key_iv: "eba2bb5f6fa5397feb112172fac34700"
server_key_version: "encryptionkey@0"
I20260619 06:45:46.288921 27068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "e58c9d8c9feb49c5b50595efd57fb1b9"
format_stamp: "Formatted at 2026-06-19 06:45:46 on dist-test-slave-5w2x"
server_key: "d0b41bc63b9b717651147fd1cd591352"
server_key_iv: "eba2bb5f6fa5397feb112172fac34700"
server_key_version: "encryptionkey@0"
I20260619 06:45:46.294811 27068 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:45:46.299237 27083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:46.301100 27068 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260619 06:45:46.301259 27068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "e58c9d8c9feb49c5b50595efd57fb1b9"
format_stamp: "Formatted at 2026-06-19 06:45:46 on dist-test-slave-5w2x"
server_key: "d0b41bc63b9b717651147fd1cd591352"
server_key_iv: "eba2bb5f6fa5397feb112172fac34700"
server_key_version: "encryptionkey@0"
I20260619 06:45:46.301491 27068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:46.323696 27068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:46.328334 27068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:46.328688 27068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:46.329653 27068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:46.331022 27068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:46.331105 27068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:46.331192 27068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:46.331234 27068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:46.364353 27068 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:36259
I20260619 06:45:46.364403 27195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:36259 every 8 connection(s)
I20260619 06:45:46.365635 27068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:45:46.370995 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 27068
I20260619 06:45:46.371261 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:45:46.371986 16701 external_mini_cluster.cc:1428] Setting key fa9e31ec11b15b5c7b3e55fbe7733978
Jun 19 06:45:46 dist-test-slave-5w2x krb5kdc[26962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851546, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:45:46.385908 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:46.368059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:43339 (local address 127.16.79.126:40233)
0619 06:45:46.368547 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:46.368551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:46.369436 (+ 885us) server_negotiation.cc:407] Connection header received
0619 06:45:46.371275 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:46.371284 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:46.371386 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:46.371516 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:46.373208 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:46.374106 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:46.375644 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:46.376397 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:46.376541 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:46.379888 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:46.379897 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:46.379905 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:46.379950 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:46.382713 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:46.383628 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:46.383632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:46.383634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:46.383746 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:46.384238 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:46.384241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:46.384243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:46.384494 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:46.384605 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:46.385499 (+ 894us) server_negotiation.cc:299] Negotiation successful
0619 06:45:46.385661 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":289,"threads_started":1}
I20260619 06:45:46.387480 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:46.368611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:40233 (local address 127.16.79.65:43339)
0619 06:45:46.369279 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:46.369322 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:46.370971 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:46.371725 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:46.371733 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:46.372310 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:46.373039 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:46.373065 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:46.374270 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:46.374274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:46.375480 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:46.375492 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:46.376626 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:46.376631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:46.376822 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:46.377491 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:46.377535 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:46.379667 (+ 2132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:46.382857 (+ 3190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:46.382868 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:46.382889 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:46.383513 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:46.383862 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:46.383870 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:46.383877 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:46.384117 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:46.384621 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:46.384631 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:46.385108 (+ 477us) client_negotiation.cc:769] Sending connection context
0619 06:45:46.385427 (+ 319us) client_negotiation.cc:240] Negotiation successful
0619 06:45:46.386002 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":255,"threads_started":1}
I20260619 06:45:46.389151 27196 heartbeater.cc:347] Connected to a master server at 127.16.79.126:40233
I20260619 06:45:46.389456 27196 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:46.390201 27196 heartbeater.cc:511] Master 127.16.79.126:40233 requested a full tablet report, sending...
I20260619 06:45:46.392347 27007 ts_manager.cc:194] Registered new tserver with Master: e58c9d8c9feb49c5b50595efd57fb1b9 (127.16.79.65:36259)
I20260619 06:45:46.393785 27007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:43339
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:45:46.423732 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:40233
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:46.755859 27202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:46.756237 27202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:46.756387 27202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:46.767928 27202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:46.768067 27202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:46.768261 27202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:45:46.780676 27202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:40233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:46.781891 27202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:46.783175 27202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:46.796054 27207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:46.796020 27208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:46.796218 27210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:46.797626 27202 server_base.cc:1034] running on GCE node
I20260619 06:45:46.798220 27202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:46.799140 27202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:46.800314 27202 hybrid_clock.cc:648] HybridClock initialized: now 1781851546800254 us; error 57 us; skew 500 ppm
I20260619 06:45:46.800625 27202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:46 dist-test-slave-5w2x krb5kdc[26962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851546, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:46.803612 27202 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:45:46.805266 27202 webserver.cc:460] Webserver started at http://127.16.79.66:44919/ using document root <none> and password file <none>
I20260619 06:45:46.805961 27202 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:46.806084 27202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:46.806411 27202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:46.809329 27202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666"
format_stamp: "Formatted at 2026-06-19 06:45:46 on dist-test-slave-5w2x"
server_key: "8908c10ca1e3affa1c9ed8567a6b1402"
server_key_iv: "257f1bd86bd621de08ddda8d4c5978a6"
server_key_version: "encryptionkey@0"
I20260619 06:45:46.810086 27202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666"
format_stamp: "Formatted at 2026-06-19 06:45:46 on dist-test-slave-5w2x"
server_key: "8908c10ca1e3affa1c9ed8567a6b1402"
server_key_iv: "257f1bd86bd621de08ddda8d4c5978a6"
server_key_version: "encryptionkey@0"
I20260619 06:45:46.815311 27202 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 06:45:46.818820 27217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:46.820091 27202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:45:46.820329 27202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666"
format_stamp: "Formatted at 2026-06-19 06:45:46 on dist-test-slave-5w2x"
server_key: "8908c10ca1e3affa1c9ed8567a6b1402"
server_key_iv: "257f1bd86bd621de08ddda8d4c5978a6"
server_key_version: "encryptionkey@0"
I20260619 06:45:46.820552 27202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:46.842314 27202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:46.846395 27202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:46.846721 27202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:46.847689 27202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:46.849068 27202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:46.849170 27202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:46.849326 27202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:46.849400 27202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:46.882344 27202 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:41889
I20260619 06:45:46.882387 27329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:41889 every 8 connection(s)
I20260619 06:45:46.883651 27202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:45:46.887960 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 27202
I20260619 06:45:46.888216 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:45:46.888777 16701 external_mini_cluster.cc:1428] Setting key a322eb268bc985d036b4f27c50413e28
Jun 19 06:45:46 dist-test-slave-5w2x krb5kdc[26962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851546, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:45:46.906275 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:46.886224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:35959 (local address 127.16.79.126:40233)
0619 06:45:46.886396 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:46.886401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:46.887643 (+ 1242us) server_negotiation.cc:407] Connection header received
0619 06:45:46.889995 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:46.889999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:46.890066 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:46.890149 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:46.892056 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:46.892805 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:46.895290 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:46.895711 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:46.895840 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:46.900057 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:46.900062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:46.900065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:46.900097 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:46.902674 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:46.903613 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:46.903616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:46.903618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:46.903734 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:46.904286 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:46.904289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:46.904291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:46.904564 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:46.904719 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:46.905690 (+ 971us) server_negotiation.cc:299] Negotiation successful
0619 06:45:46.905934 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":66}
I20260619 06:45:46.907748 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:46.886779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:40233 (local address 127.16.79.66:35959)
0619 06:45:46.887456 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:46.887492 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:46.889701 (+ 2209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:46.890416 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:46.890427 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:46.891075 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:46.891874 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:46.891905 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:46.892960 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:46.892968 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:46.895067 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:46.895085 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:46.895936 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:46.895945 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:46.896321 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:46.897093 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:46.897115 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:46.899803 (+ 2688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:46.902821 (+ 3018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:46.902833 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:46.902853 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:46.903496 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:46.903845 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:46.903853 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:46.903859 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:46.904109 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:46.904695 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:46.904706 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:46.905243 (+ 537us) client_negotiation.cc:769] Sending connection context
0619 06:45:46.905583 (+ 340us) client_negotiation.cc:240] Negotiation successful
0619 06:45:46.906215 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":260,"threads_started":1}
I20260619 06:45:46.909368 27330 heartbeater.cc:347] Connected to a master server at 127.16.79.126:40233
I20260619 06:45:46.909703 27330 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:46.910451 27330 heartbeater.cc:511] Master 127.16.79.126:40233 requested a full tablet report, sending...
I20260619 06:45:46.912039 27007 ts_manager.cc:194] Registered new tserver with Master: a6f1a53a9e104ec58ccfa6e63dfb8666 (127.16.79.66:41889)
I20260619 06:45:46.913290 27007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:35959
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:45:46.942888 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:40233
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:45:47.272565 27335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:45:47.272833 27335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:45:47.272925 27335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:45:47.280833 27335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:45:47.280949 27335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:45:47.281068 27335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:45:47.291034 27335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:40233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:45:47.292476 27335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:45:47.293717 27335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:45:47.307297 27343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:47.307349 27340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:45:47.307370 27341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:45:47.308708 27335 server_base.cc:1034] running on GCE node
I20260619 06:45:47.309338 27335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:45:47.310316 27335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:45:47.311558 27335 hybrid_clock.cc:648] HybridClock initialized: now 1781851547311517 us; error 37 us; skew 500 ppm
I20260619 06:45:47.311905 27335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:45:47 dist-test-slave-5w2x krb5kdc[26962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851547, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:45:47.315191 27335 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:45:47.316828 27335 webserver.cc:460] Webserver started at http://127.16.79.67:39523/ using document root <none> and password file <none>
I20260619 06:45:47.317631 27335 fs_manager.cc:359] Metadata directory not provided
I20260619 06:45:47.317775 27335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:45:47.318073 27335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:45:47.321120 27335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "560bc4123bf6412aa90700697d183fff"
format_stamp: "Formatted at 2026-06-19 06:45:47 on dist-test-slave-5w2x"
server_key: "ef4438a76f629fed65180b27ce6e5e55"
server_key_iv: "5cb5b9bc44695b5b516ee1dc3819c55a"
server_key_version: "encryptionkey@0"
I20260619 06:45:47.321900 27335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "560bc4123bf6412aa90700697d183fff"
format_stamp: "Formatted at 2026-06-19 06:45:47 on dist-test-slave-5w2x"
server_key: "ef4438a76f629fed65180b27ce6e5e55"
server_key_iv: "5cb5b9bc44695b5b516ee1dc3819c55a"
server_key_version: "encryptionkey@0"
I20260619 06:45:47.327360 27335 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:45:47.331274 27350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:47.332737 27335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:45:47.332875 27335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "560bc4123bf6412aa90700697d183fff"
format_stamp: "Formatted at 2026-06-19 06:45:47 on dist-test-slave-5w2x"
server_key: "ef4438a76f629fed65180b27ce6e5e55"
server_key_iv: "5cb5b9bc44695b5b516ee1dc3819c55a"
server_key_version: "encryptionkey@0"
I20260619 06:45:47.333039 27335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:45:47.349546 27335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:45:47.354080 27335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:45:47.354357 27335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:45:47.355286 27335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:45:47.356561 27335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:45:47.356643 27335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:47.356726 27335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:45:47.356769 27335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:45:47.390738 27335 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:39701
I20260619 06:45:47.390774 27462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:39701 every 8 connection(s)
I20260619 06:45:47.391978 27335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:45:47.395351 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 27335
I20260619 06:45:47.395571 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:45:47.396031 16701 external_mini_cluster.cc:1428] Setting key c56e128d4548b5c74f32210de444747f
I20260619 06:45:47.397038 27196 heartbeater.cc:503] Master 127.16.79.126:40233 was elected leader, sending a full tablet report...
Jun 19 06:45:47 dist-test-slave-5w2x krb5kdc[26962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851547, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:45:47.412957 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:47.394601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:39269 (local address 127.16.79.126:40233)
0619 06:45:47.394818 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:47.394825 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:47.396284 (+ 1459us) server_negotiation.cc:407] Connection header received
0619 06:45:47.398707 (+ 2423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:47.398712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:47.398793 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:47.398903 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:47.400631 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.401373 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.402879 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.403268 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.403406 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:47.407058 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:47.407063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:47.407066 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:47.407097 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:47.409729 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:47.410598 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:47.410606 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:47.410615 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:47.410743 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:47.411188 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:47.411192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:47.411194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:47.411469 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:47.411633 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:47.412390 (+ 757us) server_negotiation.cc:299] Negotiation successful
0619 06:45:47.412617 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 06:45:47.413923 27465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:47.395233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:40233 (local address 127.16.79.67:39269)
0619 06:45:47.396073 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:47.396121 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:47.398348 (+ 2227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:47.399192 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:47.399204 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:47.399812 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:45:47.400444 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:47.400461 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.401506 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.401509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:47.402730 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:47.402741 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.403460 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.403464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:47.403675 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:47.404448 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:45:47.404475 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:45:47.406854 (+ 2379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:45:47.409877 (+ 3023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:47.409885 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:47.409902 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:47.410471 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:47.410853 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:45:47.410859 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:45:47.410863 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:45:47.411061 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:45:47.411603 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:45:47.411611 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:45:47.411969 (+ 358us) client_negotiation.cc:769] Sending connection context
0619 06:45:47.412296 (+ 327us) client_negotiation.cc:240] Negotiation successful
0619 06:45:47.412824 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":371,"threads_started":1}
I20260619 06:45:47.415319 27463 heartbeater.cc:347] Connected to a master server at 127.16.79.126:40233
I20260619 06:45:47.415586 27463 heartbeater.cc:464] Registering TS with master...
I20260619 06:45:47.416361 27463 heartbeater.cc:511] Master 127.16.79.126:40233 requested a full tablet report, sending...
I20260619 06:45:47.417686 27006 ts_manager.cc:194] Registered new tserver with Master: 560bc4123bf6412aa90700697d183fff (127.16.79.67:39701)
I20260619 06:45:47.418598 27006 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:39269
I20260619 06:45:47.424448 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:45:47.441268 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:47.429123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44680 (local address 127.16.79.126:40233)
0619 06:45:47.429354 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:47.429363 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:47.429804 (+ 441us) server_negotiation.cc:407] Connection header received
0619 06:45:47.429942 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:47.429946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:47.430011 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:47.430096 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:47.431474 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.432274 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.433522 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.433904 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.433997 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:47.436383 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:47.436388 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:47.436391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:47.436436 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:47.438880 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:47.439455 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:47.439459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:47.439461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:47.439594 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:47.439931 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:47.439934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:47.439936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:47.440306 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:47.440487 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:47.440781 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 06:45:47.440936 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:45:47.445721 27006 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:45:47.452665 27006 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:45:47.499500 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:47.465186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:39701 (local address 127.0.0.1:60836)
0619 06:45:47.490285 (+ 25099us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:47.490344 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:47.490618 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:47.492873 (+ 2255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:47.492882 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:47.492912 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:47.493331 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:47.493344 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.494983 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.494991 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:47.497113 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:47.497137 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.498433 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.498439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:47.498671 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:47.498729 (+ 58us) client_negotiation.cc:769] Sending connection context
0619 06:45:47.498892 (+ 163us) client_negotiation.cc:240] Negotiation successful
0619 06:45:47.499092 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24733,"mutex_wait_us":5,"thread_start_us":26679,"threads_started":1}
I20260619 06:45:47.500031 27477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:47.465476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60836 (local address 127.16.79.67:39701)
0619 06:45:47.491830 (+ 26354us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:47.491840 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:47.491881 (+ 41us) server_negotiation.cc:407] Connection header received
0619 06:45:47.491986 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:47.491995 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:47.492312 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:47.492528 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:47.493574 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.494852 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.497282 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.498193 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.498341 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:47.498592 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:47.498977 (+ 385us) server_negotiation.cc:299] Negotiation successful
0619 06:45:47.499143 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26174,"thread_start_us":26018,"threads_started":1}
I20260619 06:45:47.501710 27476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:47.466078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:36259 (local address 127.0.0.1:44294)
0619 06:45:47.492248 (+ 26170us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:47.492286 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:47.492415 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:47.493309 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:47.493313 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:47.493347 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:47.493846 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:47.493855 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.495845 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.495850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:47.498443 (+ 2593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:47.498452 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.500937 (+ 2485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.500945 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:47.501098 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:47.501126 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 06:45:47.501207 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 06:45:47.501330 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26039,"mutex_wait_us":288,"spinlock_wait_cycles":896,"thread_start_us":23670,"threads_started":1}
I20260619 06:45:47.502035 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:47.466007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44788 (local address 127.16.79.66:41889)
0619 06:45:47.489544 (+ 23537us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:47.489559 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:47.492330 (+ 2771us) server_negotiation.cc:407] Connection header received
0619 06:45:47.492562 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:47.492571 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:47.492834 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:47.493029 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:47.494219 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.495704 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.498634 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.500201 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.500328 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:47.500518 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:47.501584 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0619 06:45:47.501705 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23281,"thread_start_us":25454,"threads_started":1}
I20260619 06:45:47.502467 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:47.466746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44294 (local address 127.16.79.65:36259)
0619 06:45:47.491430 (+ 24684us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:47.491442 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:47.492414 (+ 972us) server_negotiation.cc:407] Connection header received
0619 06:45:47.492600 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:47.492610 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:47.492845 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:47.493028 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:47.494031 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.495657 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.498663 (+ 3006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.500453 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.500591 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:47.500799 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:47.502000 (+ 1201us) server_negotiation.cc:299] Negotiation successful
0619 06:45:47.502168 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24469,"thread_start_us":24293,"threads_started":1}
I20260619 06:45:47.504503 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:47.465396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:41889 (local address 127.0.0.1:44788)
0619 06:45:47.492188 (+ 26792us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:47.492206 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:47.492335 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:47.493276 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:47.493284 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:47.493313 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:47.494034 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:47.494047 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.495845 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.495850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:47.498438 (+ 2588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:47.498452 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.500647 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.500654 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:47.500814 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:47.500853 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 06:45:47.500951 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 06:45:47.501092 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26628,"mutex_wait_us":75,"spinlock_wait_cycles":896,"thread_start_us":27034,"threads_started":1}
I20260619 06:45:47.504467 27398 tablet_service.cc:1467] Processing CreateTablet for tablet 1c02ed8b129b42ceb83e887d73b55690 (DEFAULT_TABLE table=test-table [id=9bcaab69de79494e9879d7afaffd1986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:47.506270 27398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c02ed8b129b42ceb83e887d73b55690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:47.506114 27131 tablet_service.cc:1467] Processing CreateTablet for tablet 1c02ed8b129b42ceb83e887d73b55690 (DEFAULT_TABLE table=test-table [id=9bcaab69de79494e9879d7afaffd1986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:47.507426 27131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c02ed8b129b42ceb83e887d73b55690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:47.509106 27265 tablet_service.cc:1467] Processing CreateTablet for tablet 1c02ed8b129b42ceb83e887d73b55690 (DEFAULT_TABLE table=test-table [id=9bcaab69de79494e9879d7afaffd1986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:45:47.510706 27265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c02ed8b129b42ceb83e887d73b55690. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:45:47.518712 27481 tablet_bootstrap.cc:492] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9: Bootstrap starting.
I20260619 06:45:47.521091 27482 tablet_bootstrap.cc:492] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff: Bootstrap starting.
I20260619 06:45:47.522696 27481 tablet_bootstrap.cc:654] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:47.524833 27481 log.cc:824] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:47.524936 27482 tablet_bootstrap.cc:654] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:47.526399 27482 log.cc:824] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:47.528239 27483 tablet_bootstrap.cc:492] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666: Bootstrap starting.
I20260619 06:45:47.528798 27481 tablet_bootstrap.cc:492] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9: No bootstrap required, opened a new log
I20260619 06:45:47.529175 27481 ts_tablet_manager.cc:1397] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260619 06:45:47.531345 27483 tablet_bootstrap.cc:654] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666: Neither blocks nor log segments found. Creating new log.
I20260619 06:45:47.532831 27483 log.cc:824] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666: Log is configured to *not* fsync() on all Append() calls
I20260619 06:45:47.534677 27482 tablet_bootstrap.cc:492] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff: No bootstrap required, opened a new log
I20260619 06:45:47.534997 27482 ts_tablet_manager.cc:1397] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.006s
I20260619 06:45:47.536530 27483 tablet_bootstrap.cc:492] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666: No bootstrap required, opened a new log
I20260619 06:45:47.536811 27483 ts_tablet_manager.cc:1397] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260619 06:45:47.538985 27481 raft_consensus.cc:348] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560bc4123bf6412aa90700697d183fff" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39701 } } peers { permanent_uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41889 } } peers { permanent_uuid: "e58c9d8c9feb49c5b50595efd57fb1b9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36259 } }
I20260619 06:45:47.539376 27481 raft_consensus.cc:374] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:47.539474 27481 raft_consensus.cc:729] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e58c9d8c9feb49c5b50595efd57fb1b9, State: Initialized, Role: FOLLOWER
I20260619 06:45:47.540105 27481 consensus_queue.cc:260] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560bc4123bf6412aa90700697d183fff" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39701 } } peers { permanent_uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41889 } } peers { permanent_uuid: "e58c9d8c9feb49c5b50595efd57fb1b9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36259 } }
I20260619 06:45:47.541778 27481 ts_tablet_manager.cc:1428] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260619 06:45:47.546339 27482 raft_consensus.cc:348] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560bc4123bf6412aa90700697d183fff" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39701 } } peers { permanent_uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41889 } } peers { permanent_uuid: "e58c9d8c9feb49c5b50595efd57fb1b9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36259 } }
I20260619 06:45:47.546464 27483 raft_consensus.cc:348] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560bc4123bf6412aa90700697d183fff" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39701 } } peers { permanent_uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41889 } } peers { permanent_uuid: "e58c9d8c9feb49c5b50595efd57fb1b9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36259 } }
I20260619 06:45:47.546799 27482 raft_consensus.cc:374] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:47.546908 27482 raft_consensus.cc:729] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 560bc4123bf6412aa90700697d183fff, State: Initialized, Role: FOLLOWER
I20260619 06:45:47.546900 27483 raft_consensus.cc:374] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:45:47.547026 27483 raft_consensus.cc:729] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f1a53a9e104ec58ccfa6e63dfb8666, State: Initialized, Role: FOLLOWER
I20260619 06:45:47.547586 27482 consensus_queue.cc:260] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560bc4123bf6412aa90700697d183fff" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39701 } } peers { permanent_uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41889 } } peers { permanent_uuid: "e58c9d8c9feb49c5b50595efd57fb1b9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36259 } }
I20260619 06:45:47.547633 27483 consensus_queue.cc:260] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560bc4123bf6412aa90700697d183fff" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39701 } } peers { permanent_uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41889 } } peers { permanent_uuid: "e58c9d8c9feb49c5b50595efd57fb1b9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36259 } }
I20260619 06:45:47.548969 27463 heartbeater.cc:503] Master 127.16.79.126:40233 was elected leader, sending a full tablet report...
I20260619 06:45:47.549403 27330 heartbeater.cc:503] Master 127.16.79.126:40233 was elected leader, sending a full tablet report...
I20260619 06:45:47.549467 27482 ts_tablet_manager.cc:1428] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260619 06:45:47.550087 27483 ts_tablet_manager.cc:1428] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260619 06:45:47.618019 27197 tablet.cc:2369] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:45:47.635672 27331 tablet.cc:2369] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:45:47.643697 27464 tablet.cc:2369] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:45:47.691004 27488 raft_consensus.cc:482] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:45:47.691321 27488 raft_consensus.cc:504] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560bc4123bf6412aa90700697d183fff" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39701 } } peers { permanent_uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41889 } } peers { permanent_uuid: "e58c9d8c9feb49c5b50595efd57fb1b9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36259 } }
I20260619 06:45:47.693109 27488 leader_election.cc:288] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 560bc4123bf6412aa90700697d183fff (127.16.79.67:39701), e58c9d8c9feb49c5b50595efd57fb1b9 (127.16.79.65:36259)
I20260619 06:45:47.700131 27477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:47.693575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:52319 (local address 127.16.79.67:39701)
0619 06:45:47.693781 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:47.693789 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:47.694068 (+ 279us) server_negotiation.cc:407] Connection header received
0619 06:45:47.694283 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:47.694289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:47.694361 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:47.694456 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:47.695063 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.696198 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.698216 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.699205 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.699376 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:47.699429 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:47.699751 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 06:45:47.699868 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 06:45:47.700325 27490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:47.693292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:39701 (local address 127.16.79.66:52319)
0619 06:45:47.693950 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:47.693974 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:47.694147 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:47.694485 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:47.694489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:47.694530 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:47.694922 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:47.694935 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.696321 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.696327 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:47.698061 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:47.698074 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.699407 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.699413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:47.699593 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:47.699630 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 06:45:47.699731 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 06:45:47.699868 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":1,"thread_start_us":505,"threads_started":1}
I20260619 06:45:47.700824 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:47.693994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:44555 (local address 127.16.79.65:36259)
0619 06:45:47.694228 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:47.694236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:47.694841 (+ 605us) server_negotiation.cc:407] Connection header received
0619 06:45:47.694957 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:47.694965 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:47.695060 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:47.695142 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:45:47.695731 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.696783 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.698711 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.699773 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.699906 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:47.699971 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:47.700460 (+ 489us) server_negotiation.cc:299] Negotiation successful
0619 06:45:47.700590 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:45:47.700878 27491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:47.693546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:36259 (local address 127.16.79.66:44555)
0619 06:45:47.694729 (+ 1183us) negotiation.cc:107] Waiting for socket to connect
0619 06:45:47.694742 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 06:45:47.694850 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:45:47.695184 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:45:47.695188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:45:47.695211 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:45:47.695572 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:47.695585 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.696930 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.696934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:47.698559 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:45:47.698572 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.699984 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.699990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:45:47.700222 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:47.700265 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 06:45:47.700411 (+ 146us) client_negotiation.cc:240] Negotiation successful
0619 06:45:47.700531 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1077,"thread_start_us":868,"threads_started":1}
I20260619 06:45:47.701512 27418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c02ed8b129b42ceb83e887d73b55690" candidate_uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560bc4123bf6412aa90700697d183fff" is_pre_election: true
I20260619 06:45:47.701745 27151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c02ed8b129b42ceb83e887d73b55690" candidate_uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e58c9d8c9feb49c5b50595efd57fb1b9" is_pre_election: true
I20260619 06:45:47.702129 27418 raft_consensus.cc:2454] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6f1a53a9e104ec58ccfa6e63dfb8666 in term 0.
I20260619 06:45:47.702242 27151 raft_consensus.cc:2454] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6f1a53a9e104ec58ccfa6e63dfb8666 in term 0.
I20260619 06:45:47.702764 27219 leader_election.cc:302] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 560bc4123bf6412aa90700697d183fff, a6f1a53a9e104ec58ccfa6e63dfb8666; no voters:
I20260619 06:45:47.703152 27488 raft_consensus.cc:2781] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:45:47.703297 27488 raft_consensus.cc:482] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:45:47.703393 27488 raft_consensus.cc:3037] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:47.705837 27488 raft_consensus.cc:504] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560bc4123bf6412aa90700697d183fff" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39701 } } peers { permanent_uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41889 } } peers { permanent_uuid: "e58c9d8c9feb49c5b50595efd57fb1b9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36259 } }
I20260619 06:45:47.706507 27488 leader_election.cc:288] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [CANDIDATE]: Term 1 election: Requested vote from peers 560bc4123bf6412aa90700697d183fff (127.16.79.67:39701), e58c9d8c9feb49c5b50595efd57fb1b9 (127.16.79.65:36259)
I20260619 06:45:47.706902 27418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c02ed8b129b42ceb83e887d73b55690" candidate_uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "560bc4123bf6412aa90700697d183fff"
I20260619 06:45:47.707002 27151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c02ed8b129b42ceb83e887d73b55690" candidate_uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e58c9d8c9feb49c5b50595efd57fb1b9"
I20260619 06:45:47.707130 27418 raft_consensus.cc:3037] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:47.707183 27151 raft_consensus.cc:3037] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:45:47.710282 27418 raft_consensus.cc:2454] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6f1a53a9e104ec58ccfa6e63dfb8666 in term 1.
I20260619 06:45:47.710282 27151 raft_consensus.cc:2454] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6f1a53a9e104ec58ccfa6e63dfb8666 in term 1.
I20260619 06:45:47.710700 27221 leader_election.cc:302] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6f1a53a9e104ec58ccfa6e63dfb8666, e58c9d8c9feb49c5b50595efd57fb1b9; no voters:
I20260619 06:45:47.711006 27488 raft_consensus.cc:2781] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:45:47.711454 27488 raft_consensus.cc:686] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [term 1 LEADER]: Becoming Leader. State: Replica: a6f1a53a9e104ec58ccfa6e63dfb8666, State: Running, Role: LEADER
I20260619 06:45:47.711925 27488 consensus_queue.cc:237] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560bc4123bf6412aa90700697d183fff" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39701 } } peers { permanent_uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41889 } } peers { permanent_uuid: "e58c9d8c9feb49c5b50595efd57fb1b9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36259 } }
I20260619 06:45:47.716745 27006 catalog_manager.cc:5495] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6f1a53a9e104ec58ccfa6e63dfb8666 (127.16.79.66). New cstate: current_term: 1 leader_uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "560bc4123bf6412aa90700697d183fff" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6f1a53a9e104ec58ccfa6e63dfb8666" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e58c9d8c9feb49c5b50595efd57fb1b9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36259 } health_report { overall_health: UNKNOWN } } }
I20260619 06:45:47.743371 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:47.737963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44790 (local address 127.16.79.66:41889)
0619 06:45:47.738169 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:47.738175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:47.738223 (+ 48us) server_negotiation.cc:407] Connection header received
0619 06:45:47.738480 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:47.738485 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:47.738575 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:47.738734 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:45:47.739098 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.739838 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.741227 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.741642 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.741776 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:47.742266 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:45:47.742774 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:45:47.742933 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:47.743013 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 06:45:47.743094 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 06:45:47.751570 27418 raft_consensus.cc:1261] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff [term 1 FOLLOWER]: Refusing update from remote peer a6f1a53a9e104ec58ccfa6e63dfb8666: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:45:47.751572 27151 raft_consensus.cc:1261] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9 [term 1 FOLLOWER]: Refusing update from remote peer a6f1a53a9e104ec58ccfa6e63dfb8666: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:45:47.752686 27492 consensus_queue.cc:1035] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "560bc4123bf6412aa90700697d183fff" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:45:47.753084 27488 consensus_queue.cc:1035] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e58c9d8c9feb49c5b50595efd57fb1b9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:45:47.774561 27497 mvcc.cc:205] Tried to move back new op lower bound from 7298463939580698624 to 7298463939431600128. Current Snapshot: MvccSnapshot[applied={T|T < 7298463939580698624}]
I20260619 06:45:47.783942 27006 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:45:47.784268 27006 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:45:47.788031 27006 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e048493c59774e229c8cb96a4c97111f: Sending DeleteTablet for 3 replicas of tablet 1c02ed8b129b42ceb83e887d73b55690
I20260619 06:45:47.789335 27265 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c02ed8b129b42ceb83e887d73b55690 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:47 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:44788
I20260619 06:45:47.789330 27398 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c02ed8b129b42ceb83e887d73b55690 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:47 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:60836
I20260619 06:45:47.789397 27131 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c02ed8b129b42ceb83e887d73b55690 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:45:47 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:44294
I20260619 06:45:47.796696 27506 tablet_replica.cc:330] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9: stopping tablet replica
I20260619 06:45:47.798015 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 27068
I20260619 06:45:47.800573 27505 tablet_replica.cc:330] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff: stopping tablet replica
I20260619 06:45:47.801331 27505 raft_consensus.cc:2229] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:45:47.802182 27505 raft_consensus.cc:2258] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:47.805362 27505 ts_tablet_manager.cc:1905] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:47.806476 27506 raft_consensus.cc:2229] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:45:47.807279 27506 raft_consensus.cc:2258] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:47.807576 27504 tablet_replica.cc:330] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666: stopping tablet replica
I20260619 06:45:47.808097 27504 raft_consensus.cc:2229] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:45:47.809716 27506 ts_tablet_manager.cc:1905] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:47.810438 27504 raft_consensus.cc:2258] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:45:47.814447 27504 ts_tablet_manager.cc:1905] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:45:47.816839 27506 ts_tablet_manager.cc:1918] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:45:47.817020 27506 log.cc:1196] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/1c02ed8b129b42ceb83e887d73b55690
I20260619 06:45:47.817557 27506 ts_tablet_manager.cc:1939] T 1c02ed8b129b42ceb83e887d73b55690 P e58c9d8c9feb49c5b50595efd57fb1b9: Deleting consensus metadata
I20260619 06:45:47.817641 27505 ts_tablet_manager.cc:1918] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:45:47.817816 27505 log.cc:1196] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/1c02ed8b129b42ceb83e887d73b55690
I20260619 06:45:47.818342 27505 ts_tablet_manager.cc:1939] T 1c02ed8b129b42ceb83e887d73b55690 P 560bc4123bf6412aa90700697d183fff: Deleting consensus metadata
I20260619 06:45:47.819515 26994 catalog_manager.cc:4841] TS e58c9d8c9feb49c5b50595efd57fb1b9 (127.16.79.65:36259): tablet 1c02ed8b129b42ceb83e887d73b55690 (table test-table [id=9bcaab69de79494e9879d7afaffd1986]) successfully deleted
I20260619 06:45:47.820015 26992 catalog_manager.cc:4841] TS 560bc4123bf6412aa90700697d183fff (127.16.79.67:39701): tablet 1c02ed8b129b42ceb83e887d73b55690 (table test-table [id=9bcaab69de79494e9879d7afaffd1986]) successfully deleted
Jun 19 06:45:47 dist-test-slave-5w2x krb5kdc[26962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:45:47.823431 27504 ts_tablet_manager.cc:1918] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:45:47.823617 27504 log.cc:1196] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/1c02ed8b129b42ceb83e887d73b55690
I20260619 06:45:47.824309 27504 ts_tablet_manager.cc:1939] T 1c02ed8b129b42ceb83e887d73b55690 P a6f1a53a9e104ec58ccfa6e63dfb8666: Deleting consensus metadata
I20260619 06:45:47.826228 26991 catalog_manager.cc:4841] TS a6f1a53a9e104ec58ccfa6e63dfb8666 (127.16.79.66:41889): tablet 1c02ed8b129b42ceb83e887d73b55690 (table test-table [id=9bcaab69de79494e9879d7afaffd1986]) successfully deleted
I20260619 06:45:47.830809 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:47.799891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44304 (local address 127.16.79.65:36259)
0619 06:45:47.800080 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:47.800086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:47.807739 (+ 7653us) server_negotiation.cc:407] Connection header received
0619 06:45:47.808030 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:47.808035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:47.808164 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:47.808251 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:47.810027 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.811194 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.813072 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.813729 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.813850 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:47.824369 (+ 10519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:47.824389 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:47.824409 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:47.824459 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:47.827705 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:47.828663 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:47.828671 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:47.828673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:47.828819 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:47.829257 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:47.829260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:47.829262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:47.829633 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:47.829853 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:47.830228 (+ 375us) server_negotiation.cc:299] Negotiation successful
0619 06:45:47.830387 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 06:45:47.831516 27191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:47.957540 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 27068
I20260619 06:45:47.980926 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 27202
Jun 19 06:45:47 dist-test-slave-5w2x krb5kdc[26962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:45:47.998230 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:47.981701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44802 (local address 127.16.79.66:41889)
0619 06:45:47.981879 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:47.981884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:47.981901 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:45:47.982116 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:47.982119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:47.982179 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:47.982259 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:47.983823 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.984847 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.986668 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:47.987086 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:47.987176 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:47.992208 (+ 5032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:47.992221 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:47.992236 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:47.992275 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:47.995203 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:47.996001 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:47.996027 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:47.996030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:47.996247 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:47.996635 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:47.996641 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:47.996645 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:47.997070 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:47.997415 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:47.997638 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 06:45:47.997857 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:45:47.998881 27325 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:48.110713 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 27202
I20260619 06:45:48.129727 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 27335
Jun 19 06:45:48 dist-test-slave-5w2x krb5kdc[26962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:45:48.144464 27477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:45:48.130485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60848 (local address 127.16.79.67:39701)
0619 06:45:48.130659 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 06:45:48.130663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:45:48.130728 (+ 65us) server_negotiation.cc:407] Connection header received
0619 06:45:48.130869 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:45:48.130873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:45:48.130931 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:45:48.131018 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:45:48.132404 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:48.133107 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:48.134333 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:45:48.134704 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:45:48.134797 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:45:48.139193 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:45:48.139208 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:45:48.139223 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:45:48.139257 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:45:48.142030 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:48.142686 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:48.142693 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:48.142695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:48.142822 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:45:48.143113 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:45:48.143116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:45:48.143118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:45:48.143443 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:45:48.143657 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0619 06:45:48.143909 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 06:45:48.144073 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:45:48.145080 27458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:48.232764 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 27335
I20260619 06:45:48.249423 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 26975
I20260619 06:45:48.250514 27037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:45:48.357774 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 26975
2026-06-19T06:45:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4375 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:45:48 dist-test-slave-5w2x krb5kdc[27514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:45:48 dist-test-slave-5w2x krb5kdc[27514](info): set up 2 sockets
Jun 19 06:45:48 dist-test-slave-5w2x krb5kdc[27514](info): commencing operation
krb5kdc: starting...
W20260619 06:45:49.461632 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:45:49 dist-test-slave-5w2x krb5kdc[27514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:45:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:45:49Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.16.79.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.16.79.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.16.79.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/rangeradmin_127.16.79.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.16.79.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/rangeradmin_127.16.79.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.16.79.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.16.79.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.16.79.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/rangerlookup_127.16.79.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.16.79.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/rangerlookup_127.16.79.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.79.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.16.79.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.79.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/HTTP_127.16.79.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.79.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/HTTP_127.16.79.84@KRBTEST.COM.keytab.
I20260619 06:45:49.787042 16701 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskf8Saqq/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/postgres -l logfile start
2026-06-19 06:45:52.211 UTC [27544] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 06:45:52.211 UTC [27544] LOG: listening on IPv4 address "127.16.79.84", port 34063
2026-06-19 06:45:52.215 UTC [27544] LOG: listening on Unix socket "/tmp/.s.PGSQL.34063"
2026-06-19 06:45:52.236 UTC [27547] LOG: database system was shut down at 2026-06-19 06:45:50 UTC
2026-06-19 06:45:52.240 UTC [27544] LOG: database system is ready to accept connections
I20260619 06:45:53.216887 16701 mini_postgres.cc:96] Postgres bound to 34063
2026-06-19 06:45:53.233 UTC [27555] FATAL: database "slave" does not exist
127.16.79.84:34063 - accepting connections
I20260619 06:45:53.234756 16701 mini_ranger.cc:161] Starting Ranger...
I20260619 06:45:53.262766 16701 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 06:45:53.730532 16701 mini_ranger.cc:87] Created ranger Postgres database
I20260619 06:45:53.730695 16701 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-admin
2026-06-19 06:45:54,027 [I] DB FLAVOR :POSTGRES
2026-06-19 06:45:54,027 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 06:45:54,027 [I] Checking connection..
2026-06-19 06:45:54,027 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 06:45:54,292 [I] Checking connection passed.
2026-06-19 06:45:54,292 [I] --------- Verifying version history table ---------
2026-06-19 06:45:54,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 06:45:54,548 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 06:45:54,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 06:45:54,804 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 06:45:54,805 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 06:45:54,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 06:45:55.043 UTC [27619] WARNING: there is no transaction in progress
2026-06-19 06:45:55,050 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 06:45:55,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 06:45:55,281 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 06:45:55,281 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 06:45:55,281 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 06:45:55,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 06:45:55,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5w2x.c.gcp-upstream.internal','N') ;"
2026-06-19 06:45:56,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 06:45:56,267 [I] Table x_portal_user does not exist in database ranger
2026-06-19 06:45:56,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 06:45:56,529 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 06:45:56,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 06:45:56,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 06:45:57,048 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 06:45:57,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 06:45:57.999 UTC [27727] WARNING: there is no transaction in progress
2026-06-19 06:45:58.016 UTC [27727] WARNING: there is no transaction in progress
2026-06-19 06:45:58.033 UTC [27727] WARNING: there is no transaction in progress
2026-06-19 06:45:58.050 UTC [27727] WARNING: there is no transaction in progress
2026-06-19 06:45:58.070 UTC [27727] WARNING: there is no transaction in progress
2026-06-19 06:45:58.091 UTC [27727] WARNING: there is no transaction in progress
2026-06-19 06:45:58.157 UTC [27727] WARNING: there is no transaction in progress
2026-06-19 06:45:58.164 UTC [27727] WARNING: there is no transaction in progress
2026-06-19 06:45:58.178 UTC [27727] WARNING: there is no transaction in progress
2026-06-19 06:45:58.188 UTC [27727] WARNING: there is no transaction in progress
2026-06-19 06:45:58.198 UTC [27727] WARNING: there is no transaction in progress
2026-06-19 06:45:58.211 UTC [27727] WARNING: there is no transaction in progress
2026-06-19 06:45:58.221 UTC [27727] WARNING: there is no transaction in progress
2026-06-19 06:45:58.227 UTC [27727] WARNING: there is no transaction in progress
2026-06-19 06:45:58.235 UTC [27727] WARNING: there is no transaction in progress
2026-06-19 06:45:58.242 UTC [27727] WARNING: there is no transaction in progress
2026-06-19 06:45:58,872 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 06:45:58,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 06:45:59,107 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 06:45:59,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 06:45:59,353 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 06:45:59,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 06:45:59,601 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 06:45:59,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 06:45:59,844 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 06:46:00,115 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5w2x.c.gcp-upstream.internal';"
2026-06-19 06:46:00,355 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 06:46:00,355 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 06:46:00,596 [I] DB_PATCHES have already been applied
I20260619 06:46:00.602856 16701 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-admin:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskf8Saqq/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 06:46:00.602979 16701 mini_ranger.cc:193] Using host: 127.16.79.84
I20260619 06:46:00.618336 16701 mini_ranger.cc:227] Ranger admin URL: http://127.16.79.84:60223
Jun 19, 2026 6:46:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 6:46:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 6:46:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 6:46:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 6:46:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 6:46:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 6:46:01 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 06:46:01.759428 16701 mini_ranger.cc:160] Time spent starting Ranger: real 8.525s user 0.000s sys 0.040s
Jun 19, 2026 6:46:22 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 6:46:22 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 19, 2026 6:46:22 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 6:46:22 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 6:46:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 6:46:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 6:46:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 6:46:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 6:46:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 6:46:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 6:46:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 6:46:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 6:46:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 6:46:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 6:46:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 6:46:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 6:46:22 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 6:46:23 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 06:46:24.024111 16701 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.16.79.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.16.79.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.16.79.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/rangerkms_127.16.79.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.16.79.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/rangerkms_127.16.79.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.79.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.16.79.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.79.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/HTTP_127.16.79.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.79.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/HTTP_127.16.79.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 06:46:24 dist-test-slave-5w2x krb5kdc[27514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851584, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 06:46:24.217371 16701 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 06:46:24.241451 16701 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 06:46:25.198143 16701 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 06:46:25.198303 16701 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-kms
I20260619 06:46:25.198433 16701 mini_ranger_kms.cc:227] Using postgres at 127.16.79.84:34063
2026-06-19 06:46:25,614 [I] DB FLAVOR :POSTGRES
2026-06-19 06:46:25,614 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 06:46:25,614 [I] Checking connection
2026-06-19 06:46:25,614 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 06:46:25,876 [I] connection success
2026-06-19 06:46:25,876 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 06:46:25,876 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 06:46:25,876 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 06:46:26,112 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 06:46:26,113 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 06:46:26,113 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 06:46:26,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:34063/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 06:46:26,364 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 06:46:26.875933 16701 mini_ranger_kms.cc:310] Created kms service
I20260619 06:46:27.095353 16701 mini_ranger_kms.cc:326] Created kudu user
I20260619 06:46:27.137858 16701 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 06:46:27 dist-test-slave-5w2x krb5kdc[27514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851584, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
I20260619 06:46:27.556413 16701 mini_ranger_kms.cc:384] Added ranger policy
I20260619 06:46:27.556731 16701 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-kms:/tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 06:46:27.556814 16701 mini_ranger_kms.cc:240] Using host: 127.16.79.84
I20260619 06:46:27.579846 16701 mini_ranger_kms.cc:276] Ranger KMS PID: 27908
I20260619 06:46:27.580036 16701 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.16.79.84:34915
Jun 19, 2026 6:46:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 6:46:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 6:46:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 6:46:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 6:46:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 6:46:29 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 19 06:46:29 dist-test-slave-5w2x krb5kdc[27514](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781851589, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.79.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:46:29.917058 16701 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.700s user 0.095s sys 0.341s
I20260619 06:46:29.917301 16701 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 06:46:29.917485 16701 mini_ranger_kms.cc:401] 127.16.79.84:34915/kms/v1/keys
Jun 19 06:46:30 dist-test-slave-5w2x krb5kdc[27514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851589, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.79.84@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
Jun 19 06:46:30 dist-test-slave-5w2x krb5kdc[27514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851589, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.79.84@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
Jun 19 06:46:32 dist-test-slave-5w2x krb5kdc[27514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:46:32.964574 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:42567
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:42877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:42567
--ranger_config_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.79.84:34915/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:46:33.298238 27958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:46:33.298547 27958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:46:33.298635 27958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:46:33.305737 27958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:46:33.305886 27958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:46:33.305933 27958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:46:33.305991 27958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:46:33.306038 27958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:46:33.316321 27958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:42877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/wal
--ranger_kms_url=127.16.79.84:34915/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:42567
--ranger_config_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:42567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:46:33.317618 27958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:46:33.318928 27958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:46:33.331787 27964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:46:33.331735 27963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:46:33.332432 27966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:46:33.332861 27958 server_base.cc:1034] running on GCE node
I20260619 06:46:33.333694 27958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:46:33.335196 27958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:46:33.336458 27958 hybrid_clock.cc:648] HybridClock initialized: now 1781851593336396 us; error 52 us; skew 500 ppm
I20260619 06:46:33.336834 27958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:46:33 dist-test-slave-5w2x krb5kdc[27514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851593, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:46:33.340240 27958 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:46:33.341862 27958 webserver.cc:460] Webserver started at http://127.16.79.126:33239/ using document root <none> and password file <none>
I20260619 06:46:33.342549 27958 fs_manager.cc:359] Metadata directory not provided
I20260619 06:46:33.342690 27958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:46:33.343010 27958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 06:46:33 dist-test-slave-5w2x krb5kdc[27514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851593, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
I20260619 06:46:33.398309 27958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "a6ac074fe8d24dd8a9709d3440667495"
format_stamp: "Formatted at 2026-06-19 06:46:33 on dist-test-slave-5w2x"
server_key: "c18f0eb0fdbafb14147b852a13b83dcf"
server_key_iv: "0140ac624262647d0684e7298559966c"
server_key_version: "kuduclusterkey@0"
I20260619 06:46:33.399150 27958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "a6ac074fe8d24dd8a9709d3440667495"
format_stamp: "Formatted at 2026-06-19 06:46:33 on dist-test-slave-5w2x"
server_key: "c18f0eb0fdbafb14147b852a13b83dcf"
server_key_iv: "0140ac624262647d0684e7298559966c"
server_key_version: "kuduclusterkey@0"
I20260619 06:46:33.404582 27958 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:46:33.423753 27973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:46:33.425452 27958 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260619 06:46:33.425637 27958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "a6ac074fe8d24dd8a9709d3440667495"
format_stamp: "Formatted at 2026-06-19 06:46:33 on dist-test-slave-5w2x"
server_key: "c18f0eb0fdbafb14147b852a13b83dcf"
server_key_iv: "0140ac624262647d0684e7298559966c"
server_key_version: "kuduclusterkey@0"
I20260619 06:46:33.425868 27958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:46:33.447191 27958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:46:33.451476 27958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:46:33.451817 27958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:46:33.473762 27958 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:42567
I20260619 06:46:33.473791 28024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:42567 every 8 connection(s)
I20260619 06:46:33.475172 27958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:46:33.479067 28025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:46:33.482542 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 27958
I20260619 06:46:33.482758 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:46:33.483393 16701 external_mini_cluster.cc:1428] Setting key eba5249ad790d13e3e51af00399217e5
I20260619 06:46:33.490159 28025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495: Bootstrap starting.
I20260619 06:46:33.493862 28025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495: Neither blocks nor log segments found. Creating new log.
I20260619 06:46:33.495016 28025 log.cc:824] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495: Log is configured to *not* fsync() on all Append() calls
I20260619 06:46:33.498507 28025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495: No bootstrap required, opened a new log
Jun 19 06:46:33 dist-test-slave-5w2x krb5kdc[27514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:46:33.508781 28028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:33.485436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60900 (local address 127.16.79.126:42567)
0619 06:46:33.486249 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0619 06:46:33.486261 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:46:33.486301 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:46:33.488710 (+ 2409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:46:33.488736 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:46:33.489153 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:46:33.489495 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:46:33.491088 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:33.492646 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:33.494474 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:33.495137 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:33.495310 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:33.499989 (+ 4679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:46:33.500000 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:46:33.500026 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:46:33.500076 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:46:33.503964 (+ 3888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:46:33.504793 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:46:33.504808 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:46:33.504817 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:46:33.505018 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:46:33.505384 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:46:33.505390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:46:33.505393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:46:33.506177 (+ 784us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:46:33.506478 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0619 06:46:33.506838 (+ 360us) server_negotiation.cc:299] Negotiation successful
0619 06:46:33.507297 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":416,"threads_started":1}
I20260619 06:46:33.509390 28025 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ac074fe8d24dd8a9709d3440667495" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42567 } }
I20260619 06:46:33.509758 28025 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:46:33.509863 28025 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ac074fe8d24dd8a9709d3440667495, State: Initialized, Role: FOLLOWER
I20260619 06:46:33.510571 28025 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ac074fe8d24dd8a9709d3440667495" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42567 } }
I20260619 06:46:33.510871 28025 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:46:33.511005 28025 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:46:33.511186 28025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:46:33.513650 28025 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ac074fe8d24dd8a9709d3440667495" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42567 } }
I20260619 06:46:33.514258 28025 leader_election.cc:302] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6ac074fe8d24dd8a9709d3440667495; no voters:
I20260619 06:46:33.514819 28025 leader_election.cc:288] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:46:33.514981 28030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:46:33.515678 28030 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [term 1 LEADER]: Becoming Leader. State: Replica: a6ac074fe8d24dd8a9709d3440667495, State: Running, Role: LEADER
I20260619 06:46:33.516360 28030 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ac074fe8d24dd8a9709d3440667495" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42567 } }
I20260619 06:46:33.516939 28025 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:46:33.519411 28031 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6ac074fe8d24dd8a9709d3440667495" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ac074fe8d24dd8a9709d3440667495" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42567 } } }
I20260619 06:46:33.519765 28031 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [sys.catalog]: This master's current role is: LEADER
I20260619 06:46:33.521706 28035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:46:33.521497 28032 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6ac074fe8d24dd8a9709d3440667495. Latest consensus state: current_term: 1 leader_uuid: "a6ac074fe8d24dd8a9709d3440667495" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ac074fe8d24dd8a9709d3440667495" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42567 } } }
I20260619 06:46:33.522029 28032 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495 [sys.catalog]: This master's current role is: LEADER
I20260619 06:46:33.528760 28035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:46:33.530874 28025 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 06:46:33.546748 28035 catalog_manager.cc:1347] Generated new cluster ID: d7484efc515b4451a53702451affa637
I20260619 06:46:33.546883 28035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:46:33.580394 28035 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:46:33.582425 28035 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:46:33.598604 28035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495: Generated new TSK 0
I20260619 06:46:33.599777 28035 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:46:34.332275 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.79.84:34915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42567
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:42877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:46:34 dist-test-slave-5w2x krb5kdc[27514](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781851594, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 06:46:34.771225 28070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:46:34.771739 28070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:46:34.771939 28070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:46:34.781713 28070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:46:34.781870 28070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:46:34.782064 28070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:46:34.792325 28070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:42877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.16.79.84:34915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:46:34.793638 28070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:46:34.795145 28070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:46:34.813429 28080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:46:34.814522 28081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:46:34.822711 28083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:46:34.823427 28070 server_base.cc:1034] running on GCE node
I20260619 06:46:34.824452 28070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:46:34.835958 28070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:46:34.837471 28070 hybrid_clock.cc:648] HybridClock initialized: now 1781851594837392 us; error 97 us; skew 500 ppm
I20260619 06:46:34.837989 28070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:46:34 dist-test-slave-5w2x krb5kdc[27514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851594, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:46:34.842762 28070 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:46:34.845911 28070 webserver.cc:460] Webserver started at http://127.16.79.65:41011/ using document root <none> and password file <none>
I20260619 06:46:34.846979 28070 fs_manager.cc:359] Metadata directory not provided
I20260619 06:46:34.847139 28070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:46:34.847561 28070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 06:46:34 dist-test-slave-5w2x krb5kdc[27514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851594, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
I20260619 06:46:34.872074 28070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "002349c350a54d119dc6a69fc7af1da8"
format_stamp: "Formatted at 2026-06-19 06:46:34 on dist-test-slave-5w2x"
server_key: "d3f7667e0ed01d71cdaa503d7273fbcf"
server_key_iv: "268547890af922474d383bfb19fa43a0"
server_key_version: "kuduclusterkey@0"
I20260619 06:46:34.873241 28070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "002349c350a54d119dc6a69fc7af1da8"
format_stamp: "Formatted at 2026-06-19 06:46:34 on dist-test-slave-5w2x"
server_key: "d3f7667e0ed01d71cdaa503d7273fbcf"
server_key_iv: "268547890af922474d383bfb19fa43a0"
server_key_version: "kuduclusterkey@0"
I20260619 06:46:34.880658 28070 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260619 06:46:34.899004 28090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:46:34.900919 28070 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 06:46:34.901170 28070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "002349c350a54d119dc6a69fc7af1da8"
format_stamp: "Formatted at 2026-06-19 06:46:34 on dist-test-slave-5w2x"
server_key: "d3f7667e0ed01d71cdaa503d7273fbcf"
server_key_iv: "268547890af922474d383bfb19fa43a0"
server_key_version: "kuduclusterkey@0"
I20260619 06:46:34.901531 28070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:46:34.936069 28070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:46:34.941594 28070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:46:34.941990 28070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:46:34.948679 28070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:46:34.950582 28070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:46:34.950726 28070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:46:34.950886 28070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:46:34.950979 28070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:46:35.019464 28070 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:33677
I20260619 06:46:35.019477 28202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:33677 every 8 connection(s)
I20260619 06:46:35.021185 28070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:46:35.030601 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 28070
I20260619 06:46:35.030807 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:46:35.031538 16701 external_mini_cluster.cc:1428] Setting key f9dd4c5424fa375be7807a175859d1e5
Jun 19 06:46:35 dist-test-slave-5w2x krb5kdc[27514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851594, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:46:35.081846 28205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:35.023990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:52627 (local address 127.16.79.126:42567)
0619 06:46:35.042525 (+ 18535us) server_negotiation.cc:206] Beginning negotiation
0619 06:46:35.042534 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:46:35.053295 (+ 10761us) server_negotiation.cc:407] Connection header received
0619 06:46:35.053398 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:46:35.053404 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:46:35.053514 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:46:35.053625 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:46:35.056016 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:35.057217 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:35.059899 (+ 2682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:35.060603 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:35.060706 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:35.071003 (+ 10297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:46:35.071011 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:46:35.071016 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:46:35.071060 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:46:35.076620 (+ 5560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:46:35.077673 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:46:35.077679 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:46:35.077683 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:46:35.077830 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:46:35.078362 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:46:35.078367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:46:35.078370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:46:35.078705 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:46:35.078825 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 06:46:35.081229 (+ 2404us) server_negotiation.cc:299] Negotiation successful
0619 06:46:35.081470 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18377,"thread_start_us":18538,"threads_started":1}
I20260619 06:46:35.083078 28206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:35.024747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:42567 (local address 127.16.79.65:52627)
0619 06:46:35.048923 (+ 24176us) negotiation.cc:107] Waiting for socket to connect
0619 06:46:35.048998 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0619 06:46:35.051711 (+ 2713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:46:35.054284 (+ 2573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:46:35.054297 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:46:35.054952 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:46:35.055793 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:46:35.055818 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:46:35.057452 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:46:35.057458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:46:35.059711 (+ 2253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:46:35.059737 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:46:35.061062 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:46:35.061070 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:46:35.062106 (+ 1036us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:35.067756 (+ 5650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:46:35.067790 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:46:35.070683 (+ 2893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:46:35.076835 (+ 6152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:46:35.076846 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:46:35.076869 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:46:35.077510 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:46:35.077971 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:46:35.077976 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:46:35.077980 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:46:35.078229 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:46:35.078914 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:46:35.078923 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:46:35.079358 (+ 435us) client_negotiation.cc:769] Sending connection context
0619 06:46:35.079668 (+ 310us) client_negotiation.cc:240] Negotiation successful
0619 06:46:35.080244 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23779,"thread_start_us":27268,"threads_started":1}
Jun 19 06:46:35 dist-test-slave-5w2x krb5kdc[27514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851594, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
I20260619 06:46:35.086406 28203 heartbeater.cc:347] Connected to a master server at 127.16.79.126:42567
I20260619 06:46:35.086761 28203 heartbeater.cc:464] Registering TS with master...
I20260619 06:46:35.087847 28203 heartbeater.cc:511] Master 127.16.79.126:42567 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
I20260619 06:46:35.090616 27990 ts_manager.cc:194] Registered new tserver with Master: 002349c350a54d119dc6a69fc7af1da8 (127.16.79.65:33677)
I20260619 06:46:35.092684 27990 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:52627
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
W20260619 06:46:35.134680 16701 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.16.79.66: real 0.100s user 0.005s sys 0.012s
I20260619 06:46:35.135675 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.79.84:34915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42567
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:42877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:46:35 dist-test-slave-5w2x krb5kdc[27514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851594, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
W20260619 06:46:35.639243 28209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:46:35.639760 28209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:46:35.639973 28209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:46:35.650876 28209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:46:35.651100 28209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:46:35.651333 28209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:46:35.667137 28209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:42877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.16.79.84:34915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:46:35.669104 28209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:46:35.670902 28209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:46:35.706453 28217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:46:35.708628 28216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:46:35.717753 28219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:46:35.718045 28209 server_base.cc:1034] running on GCE node
I20260619 06:46:35.719134 28209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:46:35.720528 28209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:46:35.721776 28209 hybrid_clock.cc:648] HybridClock initialized: now 1781851595721698 us; error 69 us; skew 500 ppm
I20260619 06:46:35.722174 28209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:46:35 dist-test-slave-5w2x krb5kdc[27514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851595, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:46:35.730046 28209 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:46:35.733763 28209 webserver.cc:460] Webserver started at http://127.16.79.66:35357/ using document root <none> and password file <none>
I20260619 06:46:35.734789 28209 fs_manager.cc:359] Metadata directory not provided
I20260619 06:46:35.735000 28209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:46:35.735392 28209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 06:46:35 dist-test-slave-5w2x krb5kdc[27514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851595, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
I20260619 06:46:35.764086 28209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "9178cecad67444329906bfe828342ef4"
format_stamp: "Formatted at 2026-06-19 06:46:35 on dist-test-slave-5w2x"
server_key: "231573464354f4aadeffadbe05abcf35"
server_key_iv: "aa6767c848e2d0bdae83444f504e4d4b"
server_key_version: "kuduclusterkey@0"
I20260619 06:46:35.765419 28209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "9178cecad67444329906bfe828342ef4"
format_stamp: "Formatted at 2026-06-19 06:46:35 on dist-test-slave-5w2x"
server_key: "231573464354f4aadeffadbe05abcf35"
server_key_iv: "aa6767c848e2d0bdae83444f504e4d4b"
server_key_version: "kuduclusterkey@0"
I20260619 06:46:35.774123 28209 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.000s
I20260619 06:46:35.793721 28233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:46:35.795797 28209 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260619 06:46:35.795996 28209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "9178cecad67444329906bfe828342ef4"
format_stamp: "Formatted at 2026-06-19 06:46:35 on dist-test-slave-5w2x"
server_key: "231573464354f4aadeffadbe05abcf35"
server_key_iv: "aa6767c848e2d0bdae83444f504e4d4b"
server_key_version: "kuduclusterkey@0"
I20260619 06:46:35.796296 28209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:46:35.827055 28209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:46:35.831933 28209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:46:35.832315 28209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:46:35.833455 28209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:46:35.835268 28209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:46:35.835363 28209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:46:35.835485 28209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:46:35.835547 28209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:46:35.872493 28209 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:33671
I20260619 06:46:35.872500 28345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:33671 every 8 connection(s)
I20260619 06:46:35.874078 28209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:46:35.875515 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 28209
I20260619 06:46:35.875705 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:46:35.876212 16701 external_mini_cluster.cc:1428] Setting key 093f596c697ede80f4d587942f81e51f
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
Jun 19 06:46:35 dist-test-slave-5w2x krb5kdc[27514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851595, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:46:35.920737 28348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:35.878028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:60243 (local address 127.16.79.126:42567)
0619 06:46:35.894372 (+ 16344us) server_negotiation.cc:206] Beginning negotiation
0619 06:46:35.894381 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:46:35.896251 (+ 1870us) server_negotiation.cc:407] Connection header received
0619 06:46:35.898278 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:46:35.898285 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:46:35.898402 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:46:35.898559 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:46:35.900716 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:35.901909 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:35.904777 (+ 2868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:35.905397 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:35.905535 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:35.913317 (+ 7782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:46:35.913324 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:46:35.913329 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:46:35.913371 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:46:35.916792 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:46:35.917846 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:46:35.917852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:46:35.917856 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:46:35.918010 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:46:35.918618 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:46:35.918623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:46:35.918626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:46:35.918974 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:46:35.919115 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 06:46:35.920088 (+ 973us) server_negotiation.cc:299] Negotiation successful
0619 06:46:35.920350 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16183,"thread_start_us":18235,"threads_started":1}
I20260619 06:46:35.922068 28349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:35.878672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:42567 (local address 127.16.79.66:60243)
0619 06:46:35.895014 (+ 16342us) negotiation.cc:107] Waiting for socket to connect
0619 06:46:35.895073 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 06:46:35.897897 (+ 2824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:46:35.898874 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:46:35.898890 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:46:35.899572 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:46:35.900513 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:46:35.900544 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:46:35.902092 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:46:35.902101 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:46:35.904592 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:46:35.904618 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:46:35.905681 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:46:35.905692 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:46:35.906065 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:35.906929 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:46:35.906971 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:46:35.912997 (+ 6026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:46:35.916975 (+ 3978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:46:35.916987 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:46:35.917010 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:46:35.917681 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:46:35.918140 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:46:35.918146 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:46:35.918149 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:46:35.918399 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:46:35.919122 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:46:35.919132 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:46:35.919637 (+ 505us) client_negotiation.cc:769] Sending connection context
0619 06:46:35.919959 (+ 322us) client_negotiation.cc:240] Negotiation successful
0619 06:46:35.920593 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16002,"thread_start_us":15783,"threads_started":1}
I20260619 06:46:35.923771 28346 heartbeater.cc:347] Connected to a master server at 127.16.79.126:42567
I20260619 06:46:35.924108 28346 heartbeater.cc:464] Registering TS with master...
I20260619 06:46:35.924831 28346 heartbeater.cc:511] Master 127.16.79.126:42567 requested a full tablet report, sending...
I20260619 06:46:35.926479 27990 ts_manager.cc:194] Registered new tserver with Master: 9178cecad67444329906bfe828342ef4 (127.16.79.66:33671)
I20260619 06:46:35.927737 27990 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:60243
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:46:35.943969 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.79.84:34915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42567
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:42877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:46:36.097891 28203 heartbeater.cc:503] Master 127.16.79.126:42567 was elected leader, sending a full tablet report...
W20260619 06:46:36.273272 28352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:46:36.273599 28352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:46:36.273753 28352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:46:36.280951 28352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:46:36.281068 28352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:46:36.281181 28352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:46:36.291107 28352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:42877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.16.79.84:34915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42567
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:46:36.292385 28352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:46:36.293744 28352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:46:36.307080 28360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:46:36.307108 28358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:46:36.307160 28357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:46:36.309507 28352 server_base.cc:1034] running on GCE node
I20260619 06:46:36.310194 28352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:46:36.311146 28352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:46:36.312376 28352 hybrid_clock.cc:648] HybridClock initialized: now 1781851596312339 us; error 51 us; skew 500 ppm
I20260619 06:46:36.312705 28352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:46:36 dist-test-slave-5w2x krb5kdc[27514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851596, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:46:36.315752 28352 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:46:36.317476 28352 webserver.cc:460] Webserver started at http://127.16.79.67:38471/ using document root <none> and password file <none>
I20260619 06:46:36.318209 28352 fs_manager.cc:359] Metadata directory not provided
I20260619 06:46:36.318358 28352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:46:36.318698 28352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 06:46:36 dist-test-slave-5w2x krb5kdc[27514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851596, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
I20260619 06:46:36.334589 28352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "661e03a6e9b446f2a27704a77d0fe767"
format_stamp: "Formatted at 2026-06-19 06:46:36 on dist-test-slave-5w2x"
server_key: "b8241510d14b95f18fd5cb7a6db5ddac"
server_key_iv: "f6d7ea74b72d3a0955269a6b0102feb6"
server_key_version: "kuduclusterkey@0"
I20260619 06:46:36.335757 28352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "661e03a6e9b446f2a27704a77d0fe767"
format_stamp: "Formatted at 2026-06-19 06:46:36 on dist-test-slave-5w2x"
server_key: "b8241510d14b95f18fd5cb7a6db5ddac"
server_key_iv: "f6d7ea74b72d3a0955269a6b0102feb6"
server_key_version: "kuduclusterkey@0"
I20260619 06:46:36.342427 28352 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260619 06:46:36.358603 28367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:46:36.360303 28352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:46:36.360504 28352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "661e03a6e9b446f2a27704a77d0fe767"
format_stamp: "Formatted at 2026-06-19 06:46:36 on dist-test-slave-5w2x"
server_key: "b8241510d14b95f18fd5cb7a6db5ddac"
server_key_iv: "f6d7ea74b72d3a0955269a6b0102feb6"
server_key_version: "kuduclusterkey@0"
I20260619 06:46:36.360761 28352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:46:36.383484 28352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:46:36.387339 28352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:46:36.387650 28352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:46:36.388767 28352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:46:36.390183 28352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:46:36.390261 28352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:46:36.390432 28352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:46:36.390484 28352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:46:36.423816 28352 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:38191
I20260619 06:46:36.423836 28479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:38191 every 8 connection(s)
I20260619 06:46:36.425204 28352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:46:36.426697 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 28352
I20260619 06:46:36.426882 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:46:36.427331 16701 external_mini_cluster.cc:1428] Setting key 920e3f3afb61bfdba5ffe150479ff786
Jun 19 06:46:36 dist-test-slave-5w2x krb5kdc[27514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851596, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:46:36.446208 28482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:36.429274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:34917 (local address 127.16.79.126:42567)
0619 06:46:36.429839 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0619 06:46:36.429846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:46:36.431010 (+ 1164us) server_negotiation.cc:407] Connection header received
0619 06:46:36.432933 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:46:36.432938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:46:36.433020 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:46:36.433112 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:46:36.434495 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.435191 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.436819 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.437235 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.437330 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:36.440509 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:46:36.440514 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:46:36.440518 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:46:36.440559 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:46:36.443566 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:46:36.444289 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:46:36.444293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:46:36.444295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:46:36.444412 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:46:36.444799 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:46:36.444802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:46:36.444804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:46:36.445059 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:46:36.445162 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 06:46:36.445768 (+ 606us) server_negotiation.cc:299] Negotiation successful
0619 06:46:36.445943 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":317,"threads_started":1}
I20260619 06:46:36.447209 28483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:36.429972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:42567 (local address 127.16.79.67:34917)
0619 06:46:36.430825 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0619 06:46:36.430876 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 06:46:36.432692 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:46:36.433320 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:46:36.433328 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:46:36.433774 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:46:36.434344 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:46:36.434360 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.435329 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.435332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:46:36.436688 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:46:36.436708 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.437417 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.437421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:46:36.437617 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:36.438221 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:46:36.438245 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:46:36.440306 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:46:36.443678 (+ 3372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:46:36.443685 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:46:36.443698 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:46:36.444122 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:46:36.444504 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:46:36.444507 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:46:36.444510 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:46:36.444703 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:46:36.445161 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:46:36.445167 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:46:36.445467 (+ 300us) client_negotiation.cc:769] Sending connection context
0619 06:46:36.445691 (+ 224us) client_negotiation.cc:240] Negotiation successful
0619 06:46:36.446101 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":318,"threads_started":1}
I20260619 06:46:36.448624 28480 heartbeater.cc:347] Connected to a master server at 127.16.79.126:42567
I20260619 06:46:36.448946 28480 heartbeater.cc:464] Registering TS with master...
I20260619 06:46:36.449673 28480 heartbeater.cc:511] Master 127.16.79.126:42567 requested a full tablet report, sending...
I20260619 06:46:36.451084 27990 ts_manager.cc:194] Registered new tserver with Master: 661e03a6e9b446f2a27704a77d0fe767 (127.16.79.67:38191)
I20260619 06:46:36.451901 27990 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:34917
I20260619 06:46:36.455868 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:46:36.473140 28482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:36.460298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52886 (local address 127.16.79.126:42567)
0619 06:46:36.460427 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 06:46:36.460432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:46:36.460909 (+ 477us) server_negotiation.cc:407] Connection header received
0619 06:46:36.461119 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:46:36.461123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:46:36.461192 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:46:36.461295 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:46:36.462713 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.463439 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.465014 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.465455 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.465597 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:36.468009 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:46:36.468014 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:46:36.468017 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:46:36.468048 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:46:36.470453 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:46:36.471142 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:46:36.471150 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:46:36.471157 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:46:36.471295 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:46:36.471701 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:46:36.471708 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:46:36.471713 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:46:36.471992 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:46:36.472260 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0619 06:46:36.472585 (+ 325us) server_negotiation.cc:299] Negotiation successful
0619 06:46:36.472812 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 06:46:36.477299 27990 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:46:36.485235 27990 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:46:36.509001 28493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:36.498340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:38191 (local address 127.0.0.1:43150)
0619 06:46:36.499365 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0619 06:46:36.499406 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 06:46:36.499603 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:46:36.500905 (+ 1302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:46:36.500912 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:46:36.500947 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:46:36.501398 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:46:36.501411 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.503802 (+ 2391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.503806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:46:36.506044 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:46:36.506055 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.507886 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.507892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:46:36.508079 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:36.508227 (+ 148us) client_negotiation.cc:769] Sending connection context
0619 06:46:36.508340 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 06:46:36.508539 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"mutex_wait_us":1,"thread_start_us":1105,"threads_started":1}
I20260619 06:46:36.509155 28496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:36.499290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43992 (local address 127.16.79.66:33671)
0619 06:46:36.500809 (+ 1519us) server_negotiation.cc:206] Beginning negotiation
0619 06:46:36.500817 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:46:36.500842 (+ 25us) server_negotiation.cc:407] Connection header received
0619 06:46:36.500932 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:46:36.500939 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:46:36.501200 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:46:36.502159 (+ 959us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:46:36.502373 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.503452 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.506431 (+ 2979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.507776 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.507892 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:36.508055 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 06:46:36.508629 (+ 574us) server_negotiation.cc:299] Negotiation successful
0619 06:46:36.508787 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1366,"thread_start_us":2013,"threads_started":1}
I20260619 06:46:36.509707 28494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:36.498713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43150 (local address 127.16.79.67:38191)
0619 06:46:36.499568 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0619 06:46:36.499576 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:46:36.499610 (+ 34us) server_negotiation.cc:407] Connection header received
0619 06:46:36.500007 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:46:36.500020 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:46:36.500348 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:46:36.500540 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:46:36.501631 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.503606 (+ 1975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.506205 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.507670 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.507806 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:36.509129 (+ 1323us) server_negotiation.cc:1035] Waiting for connection context
0619 06:46:36.509205 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 06:46:36.509375 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":751,"threads_started":1}
I20260619 06:46:36.509961 28495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:36.499265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:33671 (local address 127.0.0.1:43992)
0619 06:46:36.500040 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0619 06:46:36.500056 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:46:36.500195 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:46:36.501425 (+ 1230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:46:36.501445 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:46:36.501482 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:46:36.501970 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:46:36.501982 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.503594 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.503602 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:46:36.506280 (+ 2678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:46:36.506294 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.508235 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.508242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:46:36.508423 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:36.508459 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 06:46:36.509460 (+ 1001us) client_negotiation.cc:240] Negotiation successful
0619 06:46:36.509596 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"mutex_wait_us":6,"thread_start_us":1012,"threads_started":1}
I20260619 06:46:36.513257 28497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:36.500626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46832 (local address 127.16.79.65:33677)
0619 06:46:36.504068 (+ 3442us) server_negotiation.cc:206] Beginning negotiation
0619 06:46:36.504077 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:46:36.504101 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:46:36.504217 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:46:36.504225 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:46:36.504458 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:46:36.504640 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:46:36.505601 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.506920 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.509065 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.510445 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.510593 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:36.510832 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0619 06:46:36.512768 (+ 1936us) server_negotiation.cc:299] Negotiation successful
0619 06:46:36.512924 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3268,"thread_start_us":3102,"threads_started":1}
I20260619 06:46:36.513614 28498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:36.500897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:33677 (local address 127.0.0.1:46832)
0619 06:46:36.501982 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0619 06:46:36.502009 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 06:46:36.502115 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:46:36.504860 (+ 2745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:46:36.504868 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:46:36.504892 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:46:36.505396 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:46:36.505408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.507057 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.507063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:46:36.508852 (+ 1789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:46:36.508867 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.510640 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.512211 (+ 1571us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:46:36.512377 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:36.512411 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 06:46:36.512503 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 06:46:36.512623 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"mutex_wait_us":25,"thread_start_us":760,"threads_started":1}
I20260619 06:46:36.513178 28281 tablet_service.cc:1467] Processing CreateTablet for tablet b9503950d4bf4a1f924f76b7e3d41c41 (DEFAULT_TABLE table=test-table [id=333db622247f4d8f9aa365e51bf25309]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:46:36.513171 28415 tablet_service.cc:1467] Processing CreateTablet for tablet b9503950d4bf4a1f924f76b7e3d41c41 (DEFAULT_TABLE table=test-table [id=333db622247f4d8f9aa365e51bf25309]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:46:36.514504 28281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9503950d4bf4a1f924f76b7e3d41c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:46:36.514680 28415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9503950d4bf4a1f924f76b7e3d41c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:46:36.517156 28138 tablet_service.cc:1467] Processing CreateTablet for tablet b9503950d4bf4a1f924f76b7e3d41c41 (DEFAULT_TABLE table=test-table [id=333db622247f4d8f9aa365e51bf25309]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:46:36.518625 28138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9503950d4bf4a1f924f76b7e3d41c41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:46:36.526156 28499 tablet_bootstrap.cc:492] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4: Bootstrap starting.
I20260619 06:46:36.528683 28500 tablet_bootstrap.cc:492] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767: Bootstrap starting.
I20260619 06:46:36.529968 28499 tablet_bootstrap.cc:654] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4: Neither blocks nor log segments found. Creating new log.
I20260619 06:46:36.531884 28499 log.cc:824] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4: Log is configured to *not* fsync() on all Append() calls
I20260619 06:46:36.532352 28500 tablet_bootstrap.cc:654] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767: Neither blocks nor log segments found. Creating new log.
I20260619 06:46:36.533835 28500 log.cc:824] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767: Log is configured to *not* fsync() on all Append() calls
I20260619 06:46:36.536413 28501 tablet_bootstrap.cc:492] T b9503950d4bf4a1f924f76b7e3d41c41 P 002349c350a54d119dc6a69fc7af1da8: Bootstrap starting.
I20260619 06:46:36.537153 28500 tablet_bootstrap.cc:492] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767: No bootstrap required, opened a new log
I20260619 06:46:36.537299 28499 tablet_bootstrap.cc:492] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4: No bootstrap required, opened a new log
I20260619 06:46:36.537474 28500 ts_tablet_manager.cc:1397] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260619 06:46:36.537570 28499 ts_tablet_manager.cc:1397] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260619 06:46:36.539986 28501 tablet_bootstrap.cc:654] T b9503950d4bf4a1f924f76b7e3d41c41 P 002349c350a54d119dc6a69fc7af1da8: Neither blocks nor log segments found. Creating new log.
I20260619 06:46:36.541320 28501 log.cc:824] T b9503950d4bf4a1f924f76b7e3d41c41 P 002349c350a54d119dc6a69fc7af1da8: Log is configured to *not* fsync() on all Append() calls
I20260619 06:46:36.544365 28501 tablet_bootstrap.cc:492] T b9503950d4bf4a1f924f76b7e3d41c41 P 002349c350a54d119dc6a69fc7af1da8: No bootstrap required, opened a new log
I20260619 06:46:36.544636 28501 ts_tablet_manager.cc:1397] T b9503950d4bf4a1f924f76b7e3d41c41 P 002349c350a54d119dc6a69fc7af1da8: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260619 06:46:36.545311 28500 raft_consensus.cc:348] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e03a6e9b446f2a27704a77d0fe767" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38191 } } peers { permanent_uuid: "002349c350a54d119dc6a69fc7af1da8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33677 } } peers { permanent_uuid: "9178cecad67444329906bfe828342ef4" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33671 } }
I20260619 06:46:36.545678 28500 raft_consensus.cc:374] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:46:36.545776 28500 raft_consensus.cc:729] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 661e03a6e9b446f2a27704a77d0fe767, State: Initialized, Role: FOLLOWER
I20260619 06:46:36.546478 28500 consensus_queue.cc:260] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e03a6e9b446f2a27704a77d0fe767" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38191 } } peers { permanent_uuid: "002349c350a54d119dc6a69fc7af1da8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33677 } } peers { permanent_uuid: "9178cecad67444329906bfe828342ef4" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33671 } }
I20260619 06:46:36.547488 28499 raft_consensus.cc:348] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e03a6e9b446f2a27704a77d0fe767" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38191 } } peers { permanent_uuid: "002349c350a54d119dc6a69fc7af1da8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33677 } } peers { permanent_uuid: "9178cecad67444329906bfe828342ef4" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33671 } }
I20260619 06:46:36.547880 28499 raft_consensus.cc:374] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:46:36.548044 28499 raft_consensus.cc:729] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9178cecad67444329906bfe828342ef4, State: Initialized, Role: FOLLOWER
I20260619 06:46:36.548604 28480 heartbeater.cc:503] Master 127.16.79.126:42567 was elected leader, sending a full tablet report...
I20260619 06:46:36.548789 28499 consensus_queue.cc:260] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e03a6e9b446f2a27704a77d0fe767" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38191 } } peers { permanent_uuid: "002349c350a54d119dc6a69fc7af1da8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33677 } } peers { permanent_uuid: "9178cecad67444329906bfe828342ef4" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33671 } }
I20260619 06:46:36.549324 28500 ts_tablet_manager.cc:1428] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260619 06:46:36.550743 28346 heartbeater.cc:503] Master 127.16.79.126:42567 was elected leader, sending a full tablet report...
I20260619 06:46:36.551321 28499 ts_tablet_manager.cc:1428] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260619 06:46:36.556406 28501 raft_consensus.cc:348] T b9503950d4bf4a1f924f76b7e3d41c41 P 002349c350a54d119dc6a69fc7af1da8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e03a6e9b446f2a27704a77d0fe767" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38191 } } peers { permanent_uuid: "002349c350a54d119dc6a69fc7af1da8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33677 } } peers { permanent_uuid: "9178cecad67444329906bfe828342ef4" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33671 } }
I20260619 06:46:36.556739 28501 raft_consensus.cc:374] T b9503950d4bf4a1f924f76b7e3d41c41 P 002349c350a54d119dc6a69fc7af1da8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:46:36.556806 28501 raft_consensus.cc:729] T b9503950d4bf4a1f924f76b7e3d41c41 P 002349c350a54d119dc6a69fc7af1da8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 002349c350a54d119dc6a69fc7af1da8, State: Initialized, Role: FOLLOWER
I20260619 06:46:36.557332 28501 consensus_queue.cc:260] T b9503950d4bf4a1f924f76b7e3d41c41 P 002349c350a54d119dc6a69fc7af1da8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e03a6e9b446f2a27704a77d0fe767" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38191 } } peers { permanent_uuid: "002349c350a54d119dc6a69fc7af1da8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33677 } } peers { permanent_uuid: "9178cecad67444329906bfe828342ef4" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33671 } }
I20260619 06:46:36.558821 28501 ts_tablet_manager.cc:1428] T b9503950d4bf4a1f924f76b7e3d41c41 P 002349c350a54d119dc6a69fc7af1da8: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260619 06:46:36.626201 28347 tablet.cc:2369] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:46:36.676821 28481 tablet.cc:2369] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:46:36.805229 28204 tablet.cc:2369] T b9503950d4bf4a1f924f76b7e3d41c41 P 002349c350a54d119dc6a69fc7af1da8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:46:36.891657 28505 raft_consensus.cc:482] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:46:36.891980 28505 raft_consensus.cc:504] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e03a6e9b446f2a27704a77d0fe767" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38191 } } peers { permanent_uuid: "002349c350a54d119dc6a69fc7af1da8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33677 } } peers { permanent_uuid: "9178cecad67444329906bfe828342ef4" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33671 } }
I20260619 06:46:36.893586 28505 leader_election.cc:288] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 002349c350a54d119dc6a69fc7af1da8 (127.16.79.65:33677), 9178cecad67444329906bfe828342ef4 (127.16.79.66:33671)
I20260619 06:46:36.898985 28483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:36.894013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:33671 (local address 127.16.79.67:55611)
0619 06:46:36.894182 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 06:46:36.894207 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:46:36.894349 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:46:36.894664 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:46:36.894669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:46:36.894695 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:46:36.894882 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:46:36.894890 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.895876 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.895884 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:46:36.897186 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:46:36.897197 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.898278 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.898283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:46:36.898456 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:36.898486 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 06:46:36.898598 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 06:46:36.898726 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 06:46:36.898998 28496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:36.893979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:55611 (local address 127.16.79.66:33671)
0619 06:46:36.894364 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0619 06:46:36.894369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:46:36.894392 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:46:36.894467 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:46:36.894472 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:46:36.894538 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:46:36.894643 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:46:36.895037 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.895735 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.897351 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.898101 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.898265 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:36.898336 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 06:46:36.898623 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 06:46:36.898744 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243}
I20260619 06:46:36.900298 28301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9503950d4bf4a1f924f76b7e3d41c41" candidate_uuid: "661e03a6e9b446f2a27704a77d0fe767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9178cecad67444329906bfe828342ef4" is_pre_election: true
I20260619 06:46:36.900743 28497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:36.894073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:43179 (local address 127.16.79.65:33677)
0619 06:46:36.894289 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 06:46:36.894298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:46:36.894872 (+ 574us) server_negotiation.cc:407] Connection header received
0619 06:46:36.895108 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:46:36.895112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:46:36.895173 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:46:36.895285 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:46:36.895821 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.896670 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.898413 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.899622 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.899821 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:36.899895 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 06:46:36.900358 (+ 463us) server_negotiation.cc:299] Negotiation successful
0619 06:46:36.900507 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 06:46:36.900800 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:36.894068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:33677 (local address 127.16.79.67:43179)
0619 06:46:36.894748 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0619 06:46:36.894764 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:46:36.894949 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:46:36.895299 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:46:36.895304 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:46:36.895327 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:46:36.895651 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:46:36.895660 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.896831 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.896836 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:46:36.898239 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:46:36.898250 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.899930 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.899986 (+ 56us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:46:36.900116 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:36.900212 (+ 96us) client_negotiation.cc:769] Sending connection context
0619 06:46:36.900318 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 06:46:36.900444 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":385,"threads_started":1}
I20260619 06:46:36.900861 28301 raft_consensus.cc:2454] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 661e03a6e9b446f2a27704a77d0fe767 in term 0.
I20260619 06:46:36.901587 28370 leader_election.cc:302] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 661e03a6e9b446f2a27704a77d0fe767, 9178cecad67444329906bfe828342ef4; no voters:
I20260619 06:46:36.901961 28505 raft_consensus.cc:2781] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:46:36.901875 28158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9503950d4bf4a1f924f76b7e3d41c41" candidate_uuid: "661e03a6e9b446f2a27704a77d0fe767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "002349c350a54d119dc6a69fc7af1da8" is_pre_election: true
I20260619 06:46:36.902129 28505 raft_consensus.cc:482] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:46:36.902254 28505 raft_consensus.cc:3037] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:46:36.902499 28158 raft_consensus.cc:2454] T b9503950d4bf4a1f924f76b7e3d41c41 P 002349c350a54d119dc6a69fc7af1da8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 661e03a6e9b446f2a27704a77d0fe767 in term 0.
I20260619 06:46:36.904628 28505 raft_consensus.cc:504] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e03a6e9b446f2a27704a77d0fe767" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38191 } } peers { permanent_uuid: "002349c350a54d119dc6a69fc7af1da8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33677 } } peers { permanent_uuid: "9178cecad67444329906bfe828342ef4" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33671 } }
I20260619 06:46:36.905251 28505 leader_election.cc:288] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [CANDIDATE]: Term 1 election: Requested vote from peers 002349c350a54d119dc6a69fc7af1da8 (127.16.79.65:33677), 9178cecad67444329906bfe828342ef4 (127.16.79.66:33671)
I20260619 06:46:36.905694 28301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9503950d4bf4a1f924f76b7e3d41c41" candidate_uuid: "661e03a6e9b446f2a27704a77d0fe767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9178cecad67444329906bfe828342ef4"
I20260619 06:46:36.905689 28158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9503950d4bf4a1f924f76b7e3d41c41" candidate_uuid: "661e03a6e9b446f2a27704a77d0fe767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "002349c350a54d119dc6a69fc7af1da8"
I20260619 06:46:36.905930 28301 raft_consensus.cc:3037] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:46:36.905957 28158 raft_consensus.cc:3037] T b9503950d4bf4a1f924f76b7e3d41c41 P 002349c350a54d119dc6a69fc7af1da8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:46:36.908291 28301 raft_consensus.cc:2454] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 661e03a6e9b446f2a27704a77d0fe767 in term 1.
I20260619 06:46:36.908381 28158 raft_consensus.cc:2454] T b9503950d4bf4a1f924f76b7e3d41c41 P 002349c350a54d119dc6a69fc7af1da8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 661e03a6e9b446f2a27704a77d0fe767 in term 1.
I20260619 06:46:36.908655 28370 leader_election.cc:302] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 661e03a6e9b446f2a27704a77d0fe767, 9178cecad67444329906bfe828342ef4; no voters:
I20260619 06:46:36.908967 28505 raft_consensus.cc:2781] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:46:36.909376 28505 raft_consensus.cc:686] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [term 1 LEADER]: Becoming Leader. State: Replica: 661e03a6e9b446f2a27704a77d0fe767, State: Running, Role: LEADER
I20260619 06:46:36.909825 28505 consensus_queue.cc:237] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e03a6e9b446f2a27704a77d0fe767" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38191 } } peers { permanent_uuid: "002349c350a54d119dc6a69fc7af1da8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33677 } } peers { permanent_uuid: "9178cecad67444329906bfe828342ef4" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33671 } }
I20260619 06:46:36.914256 27989 catalog_manager.cc:5495] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 reported cstate change: term changed from 0 to 1, leader changed from <none> to 661e03a6e9b446f2a27704a77d0fe767 (127.16.79.67). New cstate: current_term: 1 leader_uuid: "661e03a6e9b446f2a27704a77d0fe767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661e03a6e9b446f2a27704a77d0fe767" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 38191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "002349c350a54d119dc6a69fc7af1da8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9178cecad67444329906bfe828342ef4" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33671 } health_report { overall_health: UNKNOWN } } }
I20260619 06:46:36.967541 28494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:36.961637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43152 (local address 127.16.79.67:38191)
0619 06:46:36.961865 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0619 06:46:36.961870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:46:36.961888 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:46:36.962130 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:46:36.962135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:46:36.962211 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:46:36.962336 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:46:36.962730 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.963437 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.965241 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:36.965691 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:36.965794 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:36.966356 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:46:36.966879 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:46:36.967002 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 06:46:36.967149 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 06:46:36.967220 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 06:46:36.975782 28301 raft_consensus.cc:1261] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4 [term 1 FOLLOWER]: Refusing update from remote peer 661e03a6e9b446f2a27704a77d0fe767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:46:36.975783 28158 raft_consensus.cc:1261] T b9503950d4bf4a1f924f76b7e3d41c41 P 002349c350a54d119dc6a69fc7af1da8 [term 1 FOLLOWER]: Refusing update from remote peer 661e03a6e9b446f2a27704a77d0fe767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:46:36.976861 28505 consensus_queue.cc:1035] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "002349c350a54d119dc6a69fc7af1da8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 33677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:46:36.977169 28509 consensus_queue.cc:1035] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9178cecad67444329906bfe828342ef4" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 33671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:46:36.990865 28512 mvcc.cc:205] Tried to move back new op lower bound from 7298464141203263488 to 7298464140946493440. Current Snapshot: MvccSnapshot[applied={T|T < 7298464141203263488}]
I20260619 06:46:37.003087 27989 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:46:37.003391 27989 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:46:37.007246 27989 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6ac074fe8d24dd8a9709d3440667495: Sending DeleteTablet for 3 replicas of tablet b9503950d4bf4a1f924f76b7e3d41c41
I20260619 06:46:37.008525 28138 tablet_service.cc:1514] Processing DeleteTablet for tablet b9503950d4bf4a1f924f76b7e3d41c41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:46:36 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:46832
I20260619 06:46:37.008533 28415 tablet_service.cc:1514] Processing DeleteTablet for tablet b9503950d4bf4a1f924f76b7e3d41c41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:46:36 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:43150
I20260619 06:46:37.009574 28281 tablet_service.cc:1514] Processing DeleteTablet for tablet b9503950d4bf4a1f924f76b7e3d41c41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:46:36 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:43992
I20260619 06:46:37.011693 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 28070
I20260619 06:46:37.020843 28524 tablet_replica.cc:330] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4: stopping tablet replica
I20260619 06:46:37.021324 28524 raft_consensus.cc:2229] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:46:37.021857 28524 raft_consensus.cc:2258] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:46:37.025800 28524 ts_tablet_manager.cc:1905] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:46:37.034056 28524 ts_tablet_manager.cc:1918] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:46:37.034346 28524 log.cc:1196] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/b9503950d4bf4a1f924f76b7e3d41c41
I20260619 06:46:37.034978 28524 ts_tablet_manager.cc:1939] T b9503950d4bf4a1f924f76b7e3d41c41 P 9178cecad67444329906bfe828342ef4: Deleting consensus metadata
I20260619 06:46:37.037606 27976 catalog_manager.cc:4841] TS 9178cecad67444329906bfe828342ef4 (127.16.79.66:33671): tablet b9503950d4bf4a1f924f76b7e3d41c41 (table test-table [id=333db622247f4d8f9aa365e51bf25309]) successfully deleted
I20260619 06:46:37.048777 28522 tablet_replica.cc:330] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767: stopping tablet replica
I20260619 06:46:37.049381 28522 raft_consensus.cc:2229] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:46:37.050391 28522 raft_consensus.cc:2258] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767 [term 1 FOLLOWER]: Raft consensus is shut down!
Jun 19 06:46:37 dist-test-slave-5w2x krb5kdc[27514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:46:37.061740 28522 ts_tablet_manager.cc:1905] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:46:37.064422 28497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:37.013548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46838 (local address 127.16.79.65:33677)
0619 06:46:37.013784 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 06:46:37.013793 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:46:37.043448 (+ 29655us) server_negotiation.cc:407] Connection header received
0619 06:46:37.043679 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:46:37.043685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:46:37.043775 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:46:37.043864 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:46:37.045623 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:37.046798 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:37.048827 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:37.049453 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:37.049560 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:37.056030 (+ 6470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:46:37.056048 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:46:37.056068 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:46:37.056117 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:46:37.060651 (+ 4534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:46:37.061603 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:46:37.061614 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:46:37.061617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:46:37.061801 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:46:37.062281 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:46:37.062286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:46:37.062289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:46:37.062751 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:46:37.063040 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0619 06:46:37.063678 (+ 638us) server_negotiation.cc:299] Negotiation successful
0619 06:46:37.063907 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 06:46:37.066478 28198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:46:37.069443 28522 ts_tablet_manager.cc:1918] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:46:37.069703 28522 log.cc:1196] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/b9503950d4bf4a1f924f76b7e3d41c41
I20260619 06:46:37.070358 28522 ts_tablet_manager.cc:1939] T b9503950d4bf4a1f924f76b7e3d41c41 P 661e03a6e9b446f2a27704a77d0fe767: Deleting consensus metadata
I20260619 06:46:37.072335 27975 catalog_manager.cc:4841] TS 661e03a6e9b446f2a27704a77d0fe767 (127.16.79.67:38191): tablet b9503950d4bf4a1f924f76b7e3d41c41 (table test-table [id=333db622247f4d8f9aa365e51bf25309]) successfully deleted
I20260619 06:46:37.181120 28523 tablet_replica.cc:330] T b9503950d4bf4a1f924f76b7e3d41c41 P 002349c350a54d119dc6a69fc7af1da8: stopping tablet replica
I20260619 06:46:37.181358 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 28070
I20260619 06:46:37.198649 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 28209
W20260619 06:46:37.198719 27975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.79.65:33677 (error 108)
I20260619 06:46:37.202561 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:37.200415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:33677 (local address 127.0.0.1:46842)
0619 06:46:37.202209 (+ 1794us) negotiation.cc:107] Waiting for socket to connect
0619 06:46:37.202334 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:33677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1672,"thread_start_us":1540,"threads_started":1}
W20260619 06:46:37.202839 27975 catalog_manager.cc:4584] TS 002349c350a54d119dc6a69fc7af1da8 (127.16.79.65:33677): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b9503950d4bf4a1f924f76b7e3d41c41: Network error: Client connection negotiation failed: client connection to 127.16.79.65:33677: connect: Connection refused (error 111)
Jun 19 06:46:37 dist-test-slave-5w2x krb5kdc[27514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:46:37.216460 28496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:37.199859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43996 (local address 127.16.79.66:33671)
0619 06:46:37.200041 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 06:46:37.200047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:46:37.200066 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:46:37.200260 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:46:37.200265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:46:37.200330 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:46:37.200438 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:46:37.201817 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:37.202878 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:37.204330 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:37.204983 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:37.205121 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:37.209377 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:46:37.209399 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:46:37.209417 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:46:37.209467 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:46:37.213310 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:46:37.214252 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:46:37.214261 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:46:37.214265 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:46:37.214419 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:46:37.214827 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:46:37.214834 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:46:37.214837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:46:37.215303 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:46:37.215604 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0619 06:46:37.215809 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 06:46:37.216031 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 06:46:37.217190 28341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:46:37.256796 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:37.256263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:33677 (local address 127.0.0.1:46852)
0619 06:46:37.256528 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 06:46:37.256632 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:33677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260619 06:46:37.258034 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:37.257607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:33677 (local address 127.0.0.1:46860)
0619 06:46:37.257800 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 06:46:37.257872 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:33677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260619 06:46:37.312755 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 28209
I20260619 06:46:37.318759 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:37.318161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:33677 (local address 127.0.0.1:46872)
0619 06:46:37.318416 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0619 06:46:37.318561 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:33677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 06:46:37.320117 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:37.319636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:33677 (local address 127.0.0.1:46884)
0619 06:46:37.319827 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 06:46:37.319929 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:33677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 06:46:37.330547 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 28352
Jun 19 06:46:37 dist-test-slave-5w2x krb5kdc[27514](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:46:37.348497 28494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:37.331384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43158 (local address 127.16.79.67:38191)
0619 06:46:37.331579 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 06:46:37.331588 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:46:37.331609 (+ 21us) server_negotiation.cc:407] Connection header received
0619 06:46:37.331785 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:46:37.331790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:46:37.331876 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:46:37.332036 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:46:37.333744 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:37.334554 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:37.336002 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:46:37.336617 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:46:37.336764 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:46:37.341785 (+ 5021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:46:37.341800 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:46:37.341815 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:46:37.341871 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:46:37.345224 (+ 3353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:46:37.346129 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:46:37.346137 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:46:37.346139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:46:37.346302 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:46:37.346724 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:46:37.346732 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:46:37.346735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:46:37.347098 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:46:37.347441 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0619 06:46:37.347796 (+ 355us) server_negotiation.cc:299] Negotiation successful
0619 06:46:37.348037 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 06:46:37.349445 28475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:46:37.401155 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:37.400606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:33677 (local address 127.0.0.1:46894)
0619 06:46:37.400860 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0619 06:46:37.400975 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:33677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260619 06:46:37.402384 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:37.401858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:33677 (local address 127.0.0.1:46904)
0619 06:46:37.402081 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 06:46:37.402170 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:33677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260619 06:46:37.477047 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 28352
I20260619 06:46:37.504751 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 27958
I20260619 06:46:37.506258 28020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:46:37.621583 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:46:37.620878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:33677 (local address 127.0.0.1:46908)
0619 06:46:37.621125 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0619 06:46:37.621293 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:33677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260619 06:46:37.621790 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 27958
I20260619 06:46:37.643711 16701 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 06:46:37.979452 16701 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 06:46:37.979714 16701 mini_ranger.cc:66] Stopping Ranger...
I20260619 06:46:38.325126 16701 mini_ranger.cc:68] Stopped Ranger
2026-06-19 06:46:38.325 UTC [27544] LOG: received smart shutdown request
2026-06-19 06:46:38.328 UTC [27544] LOG: background worker "logical replication launcher" (PID 27553) exited with exit code 1
2026-06-19 06:46:38.329 UTC [27548] LOG: shutting down
2026-06-19 06:46:38.479 UTC [27544] LOG: database system is shut down
2026-06-19T06:46:38Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50252 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:46:38 dist-test-slave-5w2x krb5kdc[28540](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:46:38 dist-test-slave-5w2x krb5kdc[28540](info): set up 2 sockets
Jun 19 06:46:38 dist-test-slave-5w2x krb5kdc[28540](info): commencing operation
krb5kdc: starting...
W20260619 06:46:39.721100 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.002s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:46:39 dist-test-slave-5w2x krb5kdc[28540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:46:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:46:39Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.16.79.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.16.79.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.16.79.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/rangeradmin_127.16.79.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.16.79.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/rangeradmin_127.16.79.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.16.79.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.16.79.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.16.79.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/rangerlookup_127.16.79.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.16.79.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/rangerlookup_127.16.79.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.79.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.16.79.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.79.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/HTTP_127.16.79.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.79.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/HTTP_127.16.79.84@KRBTEST.COM.keytab.
I20260619 06:46:40.046968 16701 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskf8Saqq/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/postgres -l logfile start
2026-06-19 06:46:42.449 UTC [28570] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 06:46:42.449 UTC [28570] LOG: listening on IPv4 address "127.16.79.84", port 37149
2026-06-19 06:46:42.453 UTC [28570] LOG: listening on Unix socket "/tmp/.s.PGSQL.37149"
2026-06-19 06:46:42.475 UTC [28573] LOG: database system was shut down at 2026-06-19 06:46:41 UTC
2026-06-19 06:46:42.478 UTC [28570] LOG: database system is ready to accept connections
I20260619 06:46:43.456188 16701 mini_postgres.cc:96] Postgres bound to 37149
2026-06-19 06:46:43.472 UTC [28581] FATAL: database "slave" does not exist
127.16.79.84:37149 - accepting connections
I20260619 06:46:43.473452 16701 mini_ranger.cc:161] Starting Ranger...
I20260619 06:46:43.498961 16701 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 06:46:43.996475 16701 mini_ranger.cc:87] Created ranger Postgres database
I20260619 06:46:43.996590 16701 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-admin
2026-06-19 06:46:44,275 [I] DB FLAVOR :POSTGRES
2026-06-19 06:46:44,275 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 06:46:44,275 [I] Checking connection..
2026-06-19 06:46:44,275 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 06:46:44,528 [I] Checking connection passed.
2026-06-19 06:46:44,529 [I] --------- Verifying version history table ---------
2026-06-19 06:46:44,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 06:46:44,779 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 06:46:44,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 06:46:45,016 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 06:46:45,016 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 06:46:45,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 06:46:45.260 UTC [28645] WARNING: there is no transaction in progress
2026-06-19 06:46:45,269 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 06:46:45,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 06:46:45,537 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 06:46:45,538 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 06:46:45,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 06:46:45,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 06:46:46,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-5w2x.c.gcp-upstream.internal','N') ;"
2026-06-19 06:46:46,275 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 06:46:46,532 [I] Table x_portal_user does not exist in database ranger
2026-06-19 06:46:46,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 06:46:46,765 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 06:46:46,766 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 06:46:46,992 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 06:46:47,226 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 06:46:47,226 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 06:46:48.194 UTC [28753] WARNING: there is no transaction in progress
2026-06-19 06:46:48.209 UTC [28753] WARNING: there is no transaction in progress
2026-06-19 06:46:48.226 UTC [28753] WARNING: there is no transaction in progress
2026-06-19 06:46:48.241 UTC [28753] WARNING: there is no transaction in progress
2026-06-19 06:46:48.257 UTC [28753] WARNING: there is no transaction in progress
2026-06-19 06:46:48.273 UTC [28753] WARNING: there is no transaction in progress
2026-06-19 06:46:48.327 UTC [28753] WARNING: there is no transaction in progress
2026-06-19 06:46:48.333 UTC [28753] WARNING: there is no transaction in progress
2026-06-19 06:46:48.347 UTC [28753] WARNING: there is no transaction in progress
2026-06-19 06:46:48.357 UTC [28753] WARNING: there is no transaction in progress
2026-06-19 06:46:48.368 UTC [28753] WARNING: there is no transaction in progress
2026-06-19 06:46:48.380 UTC [28753] WARNING: there is no transaction in progress
2026-06-19 06:46:48.390 UTC [28753] WARNING: there is no transaction in progress
2026-06-19 06:46:48.395 UTC [28753] WARNING: there is no transaction in progress
2026-06-19 06:46:48.401 UTC [28753] WARNING: there is no transaction in progress
2026-06-19 06:46:48.407 UTC [28753] WARNING: there is no transaction in progress
2026-06-19 06:46:48,986 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 06:46:48,986 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 06:46:49,221 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 06:46:49,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 06:46:49,484 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 06:46:49,485 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 06:46:49,743 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 06:46:49,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 06:46:50,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 06:46:50,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-5w2x.c.gcp-upstream.internal';"
2026-06-19 06:46:50,513 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 06:46:50,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 06:46:50,746 [I] DB_PATCHES have already been applied
I20260619 06:46:50.751830 16701 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-admin:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskf8Saqq/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 06:46:50.751927 16701 mini_ranger.cc:193] Using host: 127.16.79.84
I20260619 06:46:50.766511 16701 mini_ranger.cc:227] Ranger admin URL: http://127.16.79.84:60845
Jun 19, 2026 6:46:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 6:46:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 6:46:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 6:46:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 6:46:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 6:46:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 6:46:51 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 06:46:51.905841 16701 mini_ranger.cc:160] Time spent starting Ranger: real 8.432s user 0.000s sys 0.039s
Jun 19, 2026 6:47:12 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 6:47:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 6:47:12 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 6:47:12 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 6:47:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 6:47:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 6:47:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 6:47:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 6:47:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 6:47:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 6:47:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 6:47:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 6:47:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 6:47:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 6:47:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 6:47:12 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 6:47:12 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 6:47:13 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 06:47:14.246842 16701 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.16.79.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.16.79.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.16.79.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/rangerkms_127.16.79.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.16.79.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/rangerkms_127.16.79.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.16.79.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.16.79.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.16.79.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/HTTP_127.16.79.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.16.79.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/HTTP_127.16.79.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 06:47:14 dist-test-slave-5w2x krb5kdc[28540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851634, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 06:47:14.411319 16701 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 06:47:14.430835 16701 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 06:47:15.226421 16701 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 06:47:15.226547 16701 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-kms
I20260619 06:47:15.226622 16701 mini_ranger_kms.cc:227] Using postgres at 127.16.79.84:37149
2026-06-19 06:47:15,583 [I] DB FLAVOR :POSTGRES
2026-06-19 06:47:15,584 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 06:47:15,584 [I] Checking connection
2026-06-19 06:47:15,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 06:47:15,831 [I] connection success
2026-06-19 06:47:15,832 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 06:47:15,832 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 06:47:15,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 06:47:16,062 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 06:47:16,062 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 06:47:16,063 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 06:47:16,063 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.16.79.84:37149/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 06:47:16,343 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 06:47:16.814957 16701 mini_ranger_kms.cc:310] Created kms service
I20260619 06:47:17.012580 16701 mini_ranger_kms.cc:326] Created kudu user
I20260619 06:47:17.058286 16701 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 06:47:17 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851634, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
I20260619 06:47:17.441960 16701 mini_ranger_kms.cc:384] Added ranger policy
I20260619 06:47:17.442274 16701 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-kms:/tmp/dist-test-taskf8Saqq/build/asan/bin/postgresql.jar:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskf8Saqq/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 06:47:17.442322 16701 mini_ranger_kms.cc:240] Using host: 127.16.79.84
I20260619 06:47:17.458814 16701 mini_ranger_kms.cc:276] Ranger KMS PID: 28933
I20260619 06:47:17.458935 16701 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.16.79.84:40663
Jun 19, 2026 6:47:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 6:47:18 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 6:47:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 6:47:18 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 6:47:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 6:47:19 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 06:47:19.792259 16701 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.381s user 0.107s sys 0.264s
I20260619 06:47:19.792533 16701 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 06:47:19.792719 16701 mini_ranger_kms.cc:401] 127.16.79.84:40663/kms/v1/keys
Jun 19 06:47:19 dist-test-slave-5w2x krb5kdc[28540](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781851639, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.79.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 06:47:20 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851639, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.79.84@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
Jun 19 06:47:20 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851639, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.16.79.84@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
Jun 19 06:47:22 dist-test-slave-5w2x krb5kdc[28540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:47:22.944361 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:41619
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:41619
--ranger_config_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.79.84:40663/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:47:23.307011 28984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:23.307314 28984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:23.307411 28984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:23.314862 28984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:47:23.314978 28984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:23.315023 28984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:23.315073 28984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:47:23.315120 28984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:47:23.325539 28984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/wal
--ranger_kms_url=127.16.79.84:40663/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:41619
--ranger_config_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:41619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:23.326890 28984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:23.328228 28984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:23.341058 28990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:23.341089 28989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:23.340996 28992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:23.341652 28984 server_base.cc:1034] running on GCE node
I20260619 06:47:23.342553 28984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:23.343986 28984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:23.345224 28984 hybrid_clock.cc:648] HybridClock initialized: now 1781851643345183 us; error 53 us; skew 500 ppm
I20260619 06:47:23.345575 28984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:23 dist-test-slave-5w2x krb5kdc[28540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851643, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:23.348776 28984 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:47:23.350296 28984 webserver.cc:460] Webserver started at http://127.16.79.126:38711/ using document root <none> and password file <none>
I20260619 06:47:23.350961 28984 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:23.351104 28984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:23.351408 28984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 06:47:23 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851643, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
I20260619 06:47:23.400521 28984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "142cb03bca934dce8390b18652704813"
format_stamp: "Formatted at 2026-06-19 06:47:23 on dist-test-slave-5w2x"
server_key: "62e723b2d7b91f34f53a7f516ff5ed24"
server_key_iv: "adec61dbae859ae6220001cf1fa5b4c7"
server_key_version: "kuduclusterkey@0"
I20260619 06:47:23.401397 28984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "142cb03bca934dce8390b18652704813"
format_stamp: "Formatted at 2026-06-19 06:47:23 on dist-test-slave-5w2x"
server_key: "62e723b2d7b91f34f53a7f516ff5ed24"
server_key_iv: "adec61dbae859ae6220001cf1fa5b4c7"
server_key_version: "kuduclusterkey@0"
I20260619 06:47:23.407068 28984 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260619 06:47:23.422979 28999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:23.424612 28984 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 06:47:23.424813 28984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "142cb03bca934dce8390b18652704813"
format_stamp: "Formatted at 2026-06-19 06:47:23 on dist-test-slave-5w2x"
server_key: "62e723b2d7b91f34f53a7f516ff5ed24"
server_key_iv: "adec61dbae859ae6220001cf1fa5b4c7"
server_key_version: "kuduclusterkey@0"
I20260619 06:47:23.425062 28984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:23.451388 28984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:23.455199 28984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:23.455520 28984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:23.474157 28984 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:41619
I20260619 06:47:23.474160 29050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:41619 every 8 connection(s)
I20260619 06:47:23.475549 28984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:47:23.479406 29051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:47:23.482211 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 28984
I20260619 06:47:23.482450 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:47:23.483165 16701 external_mini_cluster.cc:1428] Setting key 48cd0998fd93351edf10557b45dfc70e
I20260619 06:47:23.492002 29051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813: Bootstrap starting.
I20260619 06:47:23.495900 29051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813: Neither blocks nor log segments found. Creating new log.
I20260619 06:47:23.497259 29051 log.cc:824] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813: Log is configured to *not* fsync() on all Append() calls
Jun 19 06:47:23 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:23.500260 29051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813: No bootstrap required, opened a new log
I20260619 06:47:23.506356 29054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:23.485280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54874 (local address 127.16.79.126:41619)
0619 06:47:23.486096 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:23.486108 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:23.486147 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:47:23.487817 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:23.487840 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:23.488334 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:23.488696 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:23.490378 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:23.491660 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:23.493540 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:23.494218 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:23.494345 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:23.499184 (+ 4839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:23.499192 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:23.499206 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:23.499239 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:23.502352 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:23.503088 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:23.503094 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:23.503100 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:23.503254 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:23.503638 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:23.503647 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:23.503649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:23.504213 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:23.504582 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:23.504942 (+ 360us) server_negotiation.cc:299] Negotiation successful
0619 06:47:23.505341 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":405,"threads_started":1}
I20260619 06:47:23.508363 29051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142cb03bca934dce8390b18652704813" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41619 } }
I20260619 06:47:23.508785 29051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:47:23.508888 29051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142cb03bca934dce8390b18652704813, State: Initialized, Role: FOLLOWER
I20260619 06:47:23.509536 29051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142cb03bca934dce8390b18652704813" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41619 } }
I20260619 06:47:23.509779 29051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:47:23.509878 29051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:47:23.509999 29051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:47:23.511981 29051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142cb03bca934dce8390b18652704813" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41619 } }
I20260619 06:47:23.512490 29051 leader_election.cc:302] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 142cb03bca934dce8390b18652704813; no voters:
I20260619 06:47:23.513025 29051 leader_election.cc:288] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:47:23.513167 29056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:47:23.513841 29056 raft_consensus.cc:686] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 1 LEADER]: Becoming Leader. State: Replica: 142cb03bca934dce8390b18652704813, State: Running, Role: LEADER
I20260619 06:47:23.514493 29056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142cb03bca934dce8390b18652704813" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41619 } }
I20260619 06:47:23.515287 29051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:47:23.517556 29057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "142cb03bca934dce8390b18652704813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142cb03bca934dce8390b18652704813" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41619 } } }
I20260619 06:47:23.517546 29058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 142cb03bca934dce8390b18652704813. Latest consensus state: current_term: 1 leader_uuid: "142cb03bca934dce8390b18652704813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142cb03bca934dce8390b18652704813" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41619 } } }
I20260619 06:47:23.518254 29058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [sys.catalog]: This master's current role is: LEADER
I20260619 06:47:23.518254 29057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [sys.catalog]: This master's current role is: LEADER
I20260619 06:47:23.521118 29062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:47:23.525856 29062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:47:23.526589 29051 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 06:47:23.543907 29062 catalog_manager.cc:1347] Generated new cluster ID: 4c23e41ca0084797933f7e385f832c88
I20260619 06:47:23.544057 29062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:47:23.574539 29062 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:47:23.576807 29062 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:47:23.590279 29062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813: Generated new TSK 0
I20260619 06:47:23.591531 29062 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:47:24.300390 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.79.84:40663/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:41619
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:47:24 dist-test-slave-5w2x krb5kdc[28540](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781851644, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260619 06:47:24.727943 29096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:24.728381 29096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:24.728559 29096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:24.740306 29096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:24.740520 29096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:24.740715 29096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:47:24.756917 29096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.16.79.84:40663/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:41619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:24.758713 29096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:24.760568 29096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:24.777554 29106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:24.779130 29107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:24.787254 29109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:24.788528 29096 server_base.cc:1034] running on GCE node
I20260619 06:47:24.789366 29096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:24.793572 29096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:24.794952 29096 hybrid_clock.cc:648] HybridClock initialized: now 1781851644794843 us; error 111 us; skew 500 ppm
I20260619 06:47:24.795436 29096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:24 dist-test-slave-5w2x krb5kdc[28540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851644, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:24.803007 29096 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:47:24.812608 29096 webserver.cc:460] Webserver started at http://127.16.79.65:34305/ using document root <none> and password file <none>
I20260619 06:47:24.813659 29096 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:24.813869 29096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:24.814261 29096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 06:47:24 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851644, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
I20260619 06:47:24.837224 29096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "1077e6bd463c401cbe13d84c1bd16294"
format_stamp: "Formatted at 2026-06-19 06:47:24 on dist-test-slave-5w2x"
server_key: "507f42c0e70d8481854d3a346a6c345b"
server_key_iv: "b4e927fa3320f51598eabc19358d6621"
server_key_version: "kuduclusterkey@0"
I20260619 06:47:24.838511 29096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "1077e6bd463c401cbe13d84c1bd16294"
format_stamp: "Formatted at 2026-06-19 06:47:24 on dist-test-slave-5w2x"
server_key: "507f42c0e70d8481854d3a346a6c345b"
server_key_iv: "b4e927fa3320f51598eabc19358d6621"
server_key_version: "kuduclusterkey@0"
I20260619 06:47:24.849144 29096 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.008s sys 0.000s
I20260619 06:47:24.875089 29116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:24.877137 29096 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.005s sys 0.004s
I20260619 06:47:24.877430 29096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "1077e6bd463c401cbe13d84c1bd16294"
format_stamp: "Formatted at 2026-06-19 06:47:24 on dist-test-slave-5w2x"
server_key: "507f42c0e70d8481854d3a346a6c345b"
server_key_iv: "b4e927fa3320f51598eabc19358d6621"
server_key_version: "kuduclusterkey@0"
I20260619 06:47:24.877779 29096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:24.935105 29096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:24.940347 29096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:24.940743 29096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:24.948829 29096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:47:24.957101 29096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:47:24.957312 29096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260619 06:47:24.957527 29096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:47:24.957657 29096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:25.025529 29096 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:34367
I20260619 06:47:25.025578 29228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:34367 every 8 connection(s)
I20260619 06:47:25.027344 29096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:47:25.033192 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 29096
I20260619 06:47:25.033424 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:47:25.034212 16701 external_mini_cluster.cc:1428] Setting key 7a5568eacd27aeabaf67101e40461e71
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
Jun 19 06:47:25 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851644, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:25.107599 29231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:25.043127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:58145 (local address 127.16.79.126:41619)
0619 06:47:25.050060 (+ 6933us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:25.050067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:25.065073 (+ 15006us) server_negotiation.cc:407] Connection header received
0619 06:47:25.067881 (+ 2808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:25.067888 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:25.067994 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:25.068078 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:25.070485 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:25.071638 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:25.074180 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:25.074835 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:25.074943 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:25.098921 (+ 23978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:25.098930 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:25.098936 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:25.098993 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:25.103219 (+ 4226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:25.104417 (+ 1198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:25.104424 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:25.104428 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:25.104590 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:25.105212 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:25.105217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:25.105220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:25.105802 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:25.105939 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:25.106960 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0619 06:47:25.107202 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6791,"thread_start_us":6658,"threads_started":1}
I20260619 06:47:25.110248 29232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:25.043956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41619 (local address 127.16.79.65:58145)
0619 06:47:25.064836 (+ 20880us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:25.064916 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:25.067489 (+ 2573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:25.068695 (+ 1206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:25.068708 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:25.069391 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:25.070264 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:25.070290 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:25.071856 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:25.071864 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:25.073985 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:25.074000 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:25.075333 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:25.075340 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:25.075663 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:25.077387 (+ 1724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:25.077424 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:25.096870 (+ 19446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:25.103463 (+ 6593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:25.103477 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:25.103500 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:25.104243 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:25.104767 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:25.104772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:25.104776 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:25.105069 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:25.106040 (+ 971us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:25.106048 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:25.106507 (+ 459us) client_negotiation.cc:769] Sending connection context
0619 06:47:25.108196 (+ 1689us) client_negotiation.cc:240] Negotiation successful
0619 06:47:25.108782 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20512,"thread_start_us":20208,"threads_started":1}
I20260619 06:47:25.112594 29229 heartbeater.cc:347] Connected to a master server at 127.16.79.126:41619
I20260619 06:47:25.112946 29229 heartbeater.cc:464] Registering TS with master...
I20260619 06:47:25.113854 29229 heartbeater.cc:511] Master 127.16.79.126:41619 requested a full tablet report, sending...
I20260619 06:47:25.116544 29016 ts_manager.cc:194] Registered new tserver with Master: 1077e6bd463c401cbe13d84c1bd16294 (127.16.79.65:34367)
I20260619 06:47:25.118613 29016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:58145
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:47:25.120225 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.79.84:40663/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:41619
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:47:25 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851644, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
Jun 19 06:47:25 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851644, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
W20260619 06:47:25.608557 29235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:25.609052 29235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:25.609256 29235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:25.620523 29235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:25.620875 29235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:25.621100 29235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:47:25.636660 29235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.16.79.84:40663/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:41619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:25.638651 29235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:25.640516 29235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:25.673995 29243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:25.677795 29246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:25.678231 29235 server_base.cc:1034] running on GCE node
W20260619 06:47:25.678972 29242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:25.679912 29235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:25.687033 29235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:25.688596 29235 hybrid_clock.cc:648] HybridClock initialized: now 1781851645688486 us; error 113 us; skew 500 ppm
I20260619 06:47:25.689083 29235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:25 dist-test-slave-5w2x krb5kdc[28540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851645, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:25.693554 29235 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:47:25.714435 29235 webserver.cc:460] Webserver started at http://127.16.79.66:43395/ using document root <none> and password file <none>
I20260619 06:47:25.715543 29235 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:25.715767 29235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:25.716272 29235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 06:47:25 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851645, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
I20260619 06:47:25.741477 29235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "cb44a34eb6404b358c47cffa27fd40a7"
format_stamp: "Formatted at 2026-06-19 06:47:25 on dist-test-slave-5w2x"
server_key: "aa6f56442539fed9fe1cf27db6e70ae2"
server_key_iv: "0408ccf4a5bf7bfc18c6123054e1d6f0"
server_key_version: "kuduclusterkey@0"
I20260619 06:47:25.743680 29235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "cb44a34eb6404b358c47cffa27fd40a7"
format_stamp: "Formatted at 2026-06-19 06:47:25 on dist-test-slave-5w2x"
server_key: "aa6f56442539fed9fe1cf27db6e70ae2"
server_key_iv: "0408ccf4a5bf7bfc18c6123054e1d6f0"
server_key_version: "kuduclusterkey@0"
I20260619 06:47:25.757624 29235 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.000s sys 0.011s
I20260619 06:47:25.787364 29252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:25.800407 29235 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.004s sys 0.000s
I20260619 06:47:25.800732 29235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "cb44a34eb6404b358c47cffa27fd40a7"
format_stamp: "Formatted at 2026-06-19 06:47:25 on dist-test-slave-5w2x"
server_key: "aa6f56442539fed9fe1cf27db6e70ae2"
server_key_iv: "0408ccf4a5bf7bfc18c6123054e1d6f0"
server_key_version: "kuduclusterkey@0"
I20260619 06:47:25.801126 29235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:25.830688 29235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:25.836335 29235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:25.836675 29235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:25.838513 29235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:47:25.840586 29235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:47:25.840687 29235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:25.840813 29235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:47:25.840873 29235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:25.895874 29235 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:45003
I20260619 06:47:25.895922 29365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:45003 every 8 connection(s)
I20260619 06:47:25.897717 29235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:47:25.902670 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 29235
I20260619 06:47:25.902904 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:47:25.903365 16701 external_mini_cluster.cc:1428] Setting key 80457c6e0f13d4f3d436d8579ccd20c8
Jun 19 06:47:25 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851645, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:25.921960 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:25.901624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:51641 (local address 127.16.79.126:41619)
0619 06:47:25.902118 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:25.902123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:25.904637 (+ 2514us) server_negotiation.cc:407] Connection header received
0619 06:47:25.906764 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:25.906770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:25.906856 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:25.906989 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:25.908734 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:25.909645 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:25.911235 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:25.911662 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:25.911805 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:25.915898 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:25.915905 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:25.915911 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:25.915950 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:25.918666 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:25.919421 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:25.919427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:25.919431 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:25.919562 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:25.920018 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:25.920025 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:25.920029 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:25.920392 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:25.920547 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:25.921327 (+ 780us) server_negotiation.cc:299] Negotiation successful
0619 06:47:25.921571 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":271,"threads_started":1}
I20260619 06:47:25.922998 29375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:25.902096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41619 (local address 127.16.79.66:51641)
0619 06:47:25.904474 (+ 2378us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:25.904533 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:25.906471 (+ 1938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:25.907238 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:25.907247 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:25.907756 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:25.908479 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:25.908523 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:25.909812 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:25.909819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:25.911089 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:25.911101 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:25.911910 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:25.911917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:25.912199 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:25.912979 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:25.913003 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:25.915629 (+ 2626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:25.918825 (+ 3196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:25.918835 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:25.918855 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:25.919279 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:25.919673 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:25.919678 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:25.919683 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:25.919898 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:25.920518 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:25.920526 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:25.920884 (+ 358us) client_negotiation.cc:769] Sending connection context
0619 06:47:25.921230 (+ 346us) client_negotiation.cc:240] Negotiation successful
0619 06:47:25.921799 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2069,"thread_start_us":1819,"threads_started":1}
I20260619 06:47:25.924551 29371 heartbeater.cc:347] Connected to a master server at 127.16.79.126:41619
I20260619 06:47:25.924872 29371 heartbeater.cc:464] Registering TS with master...
I20260619 06:47:25.925603 29371 heartbeater.cc:511] Master 127.16.79.126:41619 requested a full tablet report, sending...
I20260619 06:47:25.927228 29016 ts_manager.cc:194] Registered new tserver with Master: cb44a34eb6404b358c47cffa27fd40a7 (127.16.79.66:45003)
I20260619 06:47:25.928467 29016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:51641
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:47:25.959617 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.79.84:40663/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:41619
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:47:26.122809 29229 heartbeater.cc:503] Master 127.16.79.126:41619 was elected leader, sending a full tablet report...
W20260619 06:47:26.286098 29378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:26.286365 29378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:26.286517 29378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:26.293648 29378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:26.293749 29378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:26.293962 29378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:47:26.303347 29378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.16.79.84:40663/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:41619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:26.304677 29378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:26.306061 29378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:26.319672 29384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:26.319825 29386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:26.319855 29383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:26.320972 29378 server_base.cc:1034] running on GCE node
I20260619 06:47:26.321686 29378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:26.322664 29378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:26.323905 29378 hybrid_clock.cc:648] HybridClock initialized: now 1781851646323857 us; error 69 us; skew 500 ppm
I20260619 06:47:26.324373 29378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:26 dist-test-slave-5w2x krb5kdc[28540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851646, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:26.327550 29378 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:47:26.329172 29378 webserver.cc:460] Webserver started at http://127.16.79.67:46179/ using document root <none> and password file <none>
I20260619 06:47:26.329936 29378 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:26.330090 29378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:26.330399 29378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 06:47:26 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851646, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
I20260619 06:47:26.347129 29378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "0b092762c65f4be7aba5015481182e66"
format_stamp: "Formatted at 2026-06-19 06:47:26 on dist-test-slave-5w2x"
server_key: "5e523732075e4934f5a88e42df1370bd"
server_key_iv: "c31c26cf414c9f2a147ff54a0d05ec1c"
server_key_version: "kuduclusterkey@0"
I20260619 06:47:26.347983 29378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "0b092762c65f4be7aba5015481182e66"
format_stamp: "Formatted at 2026-06-19 06:47:26 on dist-test-slave-5w2x"
server_key: "5e523732075e4934f5a88e42df1370bd"
server_key_iv: "c31c26cf414c9f2a147ff54a0d05ec1c"
server_key_version: "kuduclusterkey@0"
I20260619 06:47:26.353330 29378 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 06:47:26.368590 29393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:26.370087 29378 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:47:26.370285 29378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "0b092762c65f4be7aba5015481182e66"
format_stamp: "Formatted at 2026-06-19 06:47:26 on dist-test-slave-5w2x"
server_key: "5e523732075e4934f5a88e42df1370bd"
server_key_iv: "c31c26cf414c9f2a147ff54a0d05ec1c"
server_key_version: "kuduclusterkey@0"
I20260619 06:47:26.370523 29378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:26.385121 29378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:26.389158 29378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:26.389460 29378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:26.390405 29378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:47:26.391842 29378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:47:26.391942 29378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:26.392089 29378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:47:26.392182 29378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260619 06:47:26.425275 29378 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:37445
I20260619 06:47:26.425311 29505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:37445 every 8 connection(s)
I20260619 06:47:26.426580 29378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:47:26.432056 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 29378
I20260619 06:47:26.432267 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:47:26.432739 16701 external_mini_cluster.cc:1428] Setting key 74781d182d74631edf82a468f5395a97
Jun 19 06:47:26 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851646, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:26.445748 29508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:26.429070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:38871 (local address 127.16.79.126:41619)
0619 06:47:26.429592 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:26.429596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:26.430393 (+ 797us) server_negotiation.cc:407] Connection header received
0619 06:47:26.432050 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:26.432055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:26.432186 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:26.432291 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:26.433906 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:26.434809 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:26.436588 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:26.437000 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:26.437130 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:26.440367 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:26.440372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:26.440375 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:26.440406 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:26.442786 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:26.443501 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:26.443506 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:26.443510 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:26.443635 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:26.444041 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:26.444048 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:26.444051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:26.444373 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:26.444523 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:26.445158 (+ 635us) server_negotiation.cc:299] Negotiation successful
0619 06:47:26.445365 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":245,"threads_started":1}
I20260619 06:47:26.446782 29509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:26.429519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41619 (local address 127.16.79.67:38871)
0619 06:47:26.430244 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:26.430285 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:26.431778 (+ 1493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:26.432559 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:26.432568 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:26.433038 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:26.433731 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:26.433749 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:26.434958 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:26.434965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:26.436448 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:26.436459 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:26.437218 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:26.437226 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:26.437460 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:26.438067 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:26.438088 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:26.440092 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:26.442931 (+ 2839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:26.442939 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:26.442955 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:26.443378 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:26.443751 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:26.443756 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:26.443760 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:26.443932 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:26.444500 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:26.444507 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:26.444824 (+ 317us) client_negotiation.cc:769] Sending connection context
0619 06:47:26.445085 (+ 261us) client_negotiation.cc:240] Negotiation successful
0619 06:47:26.445580 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":305,"threads_started":1}
I20260619 06:47:26.448354 29506 heartbeater.cc:347] Connected to a master server at 127.16.79.126:41619
I20260619 06:47:26.448609 29506 heartbeater.cc:464] Registering TS with master...
I20260619 06:47:26.449221 29506 heartbeater.cc:511] Master 127.16.79.126:41619 requested a full tablet report, sending...
I20260619 06:47:26.450541 29016 ts_manager.cc:194] Registered new tserver with Master: 0b092762c65f4be7aba5015481182e66 (127.16.79.67:37445)
I20260619 06:47:26.451367 29016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:38871
I20260619 06:47:26.460415 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:47:26.462186 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 29096
Jun 19 06:47:26 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:47:26.479560 29511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:26.463008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45830 (local address 127.16.79.65:34367)
0619 06:47:26.463914 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:26.463921 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:26.463947 (+ 26us) server_negotiation.cc:407] Connection header received
0619 06:47:26.464001 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:26.464007 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:26.464371 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:26.464559 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:26.466146 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:26.467129 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:26.468566 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:26.469120 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:26.469263 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:26.473607 (+ 4344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:26.473618 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:26.473631 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:26.473697 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:26.476695 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:26.477424 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:26.477431 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:26.477433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:26.477577 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:26.477957 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:26.477963 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:26.477967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:26.478306 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:26.478615 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:26.478933 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 06:47:26.479209 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":376,"threads_started":1}
I20260619 06:47:26.480907 29224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:26.598986 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 29096
I20260619 06:47:26.615749 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 29235
Jun 19 06:47:26 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:47:26.634755 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:26.616812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34372 (local address 127.16.79.66:45003)
0619 06:47:26.617358 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:26.617365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:26.617391 (+ 26us) server_negotiation.cc:407] Connection header received
0619 06:47:26.617453 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:26.617459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:26.617676 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:26.617829 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:26.619548 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:26.620654 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:26.622962 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:26.623542 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:26.623644 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:26.629387 (+ 5743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:26.629401 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:26.629415 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:26.629451 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:26.632190 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:26.632928 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:26.632935 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:26.632937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:26.633070 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:26.633402 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:26.633406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:26.633408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:26.633742 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:26.633959 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:26.634258 (+ 299us) server_negotiation.cc:299] Negotiation successful
0619 06:47:26.634465 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":273,"threads_started":1}
I20260619 06:47:26.636075 29360 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:26.717998 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 29235
I20260619 06:47:26.733582 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 29378
Jun 19 06:47:26 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:47:26.751727 29515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:26.734564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45438 (local address 127.16.79.67:37445)
0619 06:47:26.735169 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:26.735175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:26.735203 (+ 28us) server_negotiation.cc:407] Connection header received
0619 06:47:26.735259 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:26.735266 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:26.735496 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:26.735665 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:26.737131 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:26.738189 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:26.739528 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:26.740329 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:26.740451 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:26.745114 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:26.745130 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:26.745148 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:26.745212 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:26.748315 (+ 3103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:26.749500 (+ 1185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:26.749508 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:26.749510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:26.749651 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:26.750047 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:26.750053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:26.750056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:26.750422 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:26.750748 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:26.751084 (+ 336us) server_negotiation.cc:299] Negotiation successful
0619 06:47:26.751360 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":328,"threads_started":1}
I20260619 06:47:26.753108 29501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:26.836280 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 29378
I20260619 06:47:26.852001 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 28984
I20260619 06:47:26.853502 29046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:26.960640 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 28984
I20260619 06:47:26.983179 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:41619
--webserver_interface=127.16.79.126
--webserver_port=38711
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:41619
--ranger_config_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.79.84:40663/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.79.84:40663/kms with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:47:27.310736 29518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:27.311074 29518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:27.311225 29518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:27.318596 29518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:47:27.318706 29518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:27.318813 29518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:27.318902 29518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:47:27.318954 29518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:47:27.329157 29518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.79.84:40663/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:41619
--ranger_config_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:41619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=38711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:27.330435 29518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:27.331792 29518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:27.344118 29524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:27.344197 29526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:27.344230 29523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:27.345106 29518 server_base.cc:1034] running on GCE node
I20260619 06:47:27.345958 29518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:27.347432 29518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:27.348697 29518 hybrid_clock.cc:648] HybridClock initialized: now 1781851647348657 us; error 37 us; skew 500 ppm
I20260619 06:47:27.349031 29518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:27 dist-test-slave-5w2x krb5kdc[28540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851647, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:27.352316 29518 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:47:27.353909 29518 webserver.cc:460] Webserver started at http://127.16.79.126:38711/ using document root <none> and password file <none>
I20260619 06:47:27.354550 29518 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:27.354655 29518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 06:47:27 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851647, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
I20260619 06:47:27.438437 29518 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260619 06:47:27.441636 29534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:27.443249 29518 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:47:27.443470 29518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "142cb03bca934dce8390b18652704813"
format_stamp: "Formatted at 2026-06-19 06:47:23 on dist-test-slave-5w2x"
server_key: "62e723b2d7b91f34f53a7f516ff5ed24"
server_key_iv: "adec61dbae859ae6220001cf1fa5b4c7"
server_key_version: "kuduclusterkey@0"
I20260619 06:47:27.444593 29518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:27.473913 29518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:27.478122 29518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:27.478432 29518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:27.497018 29518 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:41619
I20260619 06:47:27.497068 29585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:41619 every 8 connection(s)
I20260619 06:47:27.498512 29518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:47:27.505390 29586 sys_catalog.cc:263] Verifying existing consensus state
I20260619 06:47:27.506922 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 29518
I20260619 06:47:27.507730 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.79.84:40663/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:34367
--local_ip_for_outbound_sockets=127.16.79.65
--tserver_master_addrs=127.16.79.126:41619
--webserver_port=34305
--webserver_interface=127.16.79.65
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.79.84:40663/kms with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:47:27.508790 29586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813: Bootstrap starting.
I20260619 06:47:27.527415 29586 log.cc:824] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:27.536492 29586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:47:27.537020 29586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813: Bootstrap complete.
I20260619 06:47:27.546262 29586 raft_consensus.cc:348] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142cb03bca934dce8390b18652704813" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41619 } }
I20260619 06:47:27.547076 29586 raft_consensus.cc:729] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142cb03bca934dce8390b18652704813, State: Initialized, Role: FOLLOWER
I20260619 06:47:27.547745 29586 consensus_queue.cc:260] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142cb03bca934dce8390b18652704813" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41619 } }
I20260619 06:47:27.548012 29586 raft_consensus.cc:388] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:47:27.548172 29586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:47:27.548396 29586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 1 FOLLOWER]: Advancing to term 2
I20260619 06:47:27.552471 29586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142cb03bca934dce8390b18652704813" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41619 } }
I20260619 06:47:27.553000 29586 leader_election.cc:302] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 142cb03bca934dce8390b18652704813; no voters:
I20260619 06:47:27.553640 29586 leader_election.cc:288] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 06:47:27.553776 29590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 06:47:27.555094 29590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [term 2 LEADER]: Becoming Leader. State: Replica: 142cb03bca934dce8390b18652704813, State: Running, Role: LEADER
I20260619 06:47:27.555622 29590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142cb03bca934dce8390b18652704813" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41619 } }
I20260619 06:47:27.556053 29586 sys_catalog.cc:564] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:47:27.558192 29591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "142cb03bca934dce8390b18652704813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142cb03bca934dce8390b18652704813" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41619 } } }
I20260619 06:47:27.558634 29591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [sys.catalog]: This master's current role is: LEADER
I20260619 06:47:27.562469 29592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 142cb03bca934dce8390b18652704813. Latest consensus state: current_term: 2 leader_uuid: "142cb03bca934dce8390b18652704813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142cb03bca934dce8390b18652704813" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 41619 } } }
I20260619 06:47:27.562806 29592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813 [sys.catalog]: This master's current role is: LEADER
I20260619 06:47:27.564060 29596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:47:27.569621 29596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:47:27.570986 29596 catalog_manager.cc:1259] Loaded cluster ID: 4c23e41ca0084797933f7e385f832c88
I20260619 06:47:27.571122 29596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:47:27.575130 29596 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:47:27.576560 29596 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813: Loaded TSK: 0
I20260619 06:47:27.578087 29596 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:47:27.581238 29586 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260619 06:47:27.965451 29588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:27.965911 29588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:27.966116 29588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:27.976912 29588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:27.977073 29588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:27.977270 29588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:47:27.992218 29588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.79.84:40663/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:34367
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=34305
--tserver_master_addrs=127.16.79.126:41619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:27.993942 29588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:27.995728 29588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:28.030649 29624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:28.032735 29623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:28.038400 29626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:28.039331 29588 server_base.cc:1034] running on GCE node
I20260619 06:47:28.040037 29588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:28.041157 29588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:28.042385 29588 hybrid_clock.cc:648] HybridClock initialized: now 1781851648042336 us; error 45 us; skew 500 ppm
I20260619 06:47:28.042722 29588 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:28 dist-test-slave-5w2x krb5kdc[28540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851648, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:28.046466 29588 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:47:28.048446 29588 webserver.cc:460] Webserver started at http://127.16.79.65:34305/ using document root <none> and password file <none>
I20260619 06:47:28.049474 29588 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:28.049631 29588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 06:47:28 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851648, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
I20260619 06:47:28.072664 29588 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:47:28.075766 29634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:28.081120 29588 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260619 06:47:28.081326 29588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "1077e6bd463c401cbe13d84c1bd16294"
format_stamp: "Formatted at 2026-06-19 06:47:24 on dist-test-slave-5w2x"
server_key: "507f42c0e70d8481854d3a346a6c345b"
server_key_iv: "b4e927fa3320f51598eabc19358d6621"
server_key_version: "kuduclusterkey@0"
I20260619 06:47:28.082762 29588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:28.108546 29588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:28.113644 29588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:28.113978 29588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:28.115180 29588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:47:28.116932 29588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:47:28.117049 29588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:28.117158 29588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:47:28.117235 29588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:28.173331 29588 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:34367
I20260619 06:47:28.173379 29747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:34367 every 8 connection(s)
I20260619 06:47:28.174768 29588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:47:28.179458 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 29588
I20260619 06:47:28.180326 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.79.84:40663/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:45003
--local_ip_for_outbound_sockets=127.16.79.66
--tserver_master_addrs=127.16.79.126:41619
--webserver_port=43395
--webserver_interface=127.16.79.66
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.79.84:40663/kms with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:47:28 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851648, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:28.204711 29751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:28.178466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41619 (local address 127.16.79.65:39617)
0619 06:47:28.179221 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:28.179255 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:28.181475 (+ 2220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:28.182894 (+ 1419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:28.182907 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:28.183410 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:28.183950 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:28.183967 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:28.185688 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:28.185697 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:28.187920 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:28.187936 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:28.189762 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:28.189770 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:28.190076 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:28.190846 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:28.190879 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:28.193914 (+ 3035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:28.198716 (+ 4802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:28.198727 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:28.198746 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:28.199405 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:28.199975 (+ 570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:28.199983 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:28.199989 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:28.200263 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:28.201516 (+ 1253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:28.201528 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:28.202016 (+ 488us) client_negotiation.cc:769] Sending connection context
0619 06:47:28.202345 (+ 329us) client_negotiation.cc:240] Negotiation successful
0619 06:47:28.203008 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":1597,"threads_started":1}
I20260619 06:47:28.207336 29750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:28.178221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:39617 (local address 127.16.79.126:41619)
0619 06:47:28.178941 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:28.178958 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:28.179409 (+ 451us) server_negotiation.cc:407] Connection header received
0619 06:47:28.182055 (+ 2646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:28.182068 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:28.182403 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:28.182769 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:28.184189 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:28.185500 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:28.188111 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:28.189187 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:28.189373 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:28.194217 (+ 4844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:28.194229 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:28.194251 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:28.194313 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:28.198529 (+ 4216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:28.199583 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:28.199594 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:28.199605 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:28.199827 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:28.200413 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:28.200421 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:28.200426 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:28.201218 (+ 792us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:28.201422 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:28.205406 (+ 3984us) server_negotiation.cc:299] Negotiation successful
0619 06:47:28.205903 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":271,"threads_started":1}
I20260619 06:47:28.210843 29748 heartbeater.cc:347] Connected to a master server at 127.16.79.126:41619
I20260619 06:47:28.211160 29748 heartbeater.cc:464] Registering TS with master...
I20260619 06:47:28.211985 29748 heartbeater.cc:511] Master 127.16.79.126:41619 requested a full tablet report, sending...
W20260619 06:47:28.215149 29748 heartbeater.cc:644] Failed to heartbeat to 127.16.79.126:41619 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 06:47:28.215389 29748 heartbeater.cc:464] Registering TS with master...
I20260619 06:47:28.215646 29748 heartbeater.cc:511] Master 127.16.79.126:41619 requested a full tablet report, sending...
I20260619 06:47:28.216894 29748 heartbeater.cc:464] Registering TS with master...
I20260619 06:47:28.217149 29748 heartbeater.cc:511] Master 127.16.79.126:41619 requested a full tablet report, sending...
W20260619 06:47:28.218154 29748 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 06:47:28.620244 29752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:28.620569 29752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:28.620752 29752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:28.628412 29752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:28.628556 29752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:28.628731 29752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:47:28.638978 29752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.79.84:40663/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:45003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=43395
--tserver_master_addrs=127.16.79.126:41619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:28.640622 29752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:28.641963 29752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:28.655893 29765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:28.656105 29767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:28.656507 29764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:28.657480 29752 server_base.cc:1034] running on GCE node
I20260619 06:47:28.658145 29752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:28.659253 29752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:28.660483 29752 hybrid_clock.cc:648] HybridClock initialized: now 1781851648660412 us; error 60 us; skew 500 ppm
I20260619 06:47:28.660846 29752 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:28 dist-test-slave-5w2x krb5kdc[28540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851648, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:28.664016 29752 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:47:28.665766 29752 webserver.cc:460] Webserver started at http://127.16.79.66:43395/ using document root <none> and password file <none>
I20260619 06:47:28.666513 29752 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:28.666656 29752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 06:47:28 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851648, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
I20260619 06:47:28.690462 29752 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:47:28.701401 29777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:28.703531 29752 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.005s sys 0.004s
I20260619 06:47:28.703717 29752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "cb44a34eb6404b358c47cffa27fd40a7"
format_stamp: "Formatted at 2026-06-19 06:47:25 on dist-test-slave-5w2x"
server_key: "aa6f56442539fed9fe1cf27db6e70ae2"
server_key_iv: "0408ccf4a5bf7bfc18c6123054e1d6f0"
server_key_version: "kuduclusterkey@0"
I20260619 06:47:28.705228 29752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:28.728350 29752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:28.733707 29752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:28.734011 29752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:28.735255 29752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:47:28.737005 29752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:47:28.737119 29752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:28.737217 29752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:47:28.737293 29752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:28.804453 29752 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:45003
I20260619 06:47:28.804508 29889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:45003 every 8 connection(s)
I20260619 06:47:28.806492 29752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:47:28.811673 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 29752
I20260619 06:47:28.812520 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.16.79.84:40663/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:37445
--local_ip_for_outbound_sockets=127.16.79.67
--tserver_master_addrs=127.16.79.126:41619
--webserver_port=46179
--webserver_interface=127.16.79.67
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.16.79.84:40663/kms with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:47:28 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851648, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:28.849867 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:28.810513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41619 (local address 127.16.79.66:46429)
0619 06:47:28.816978 (+ 6465us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:28.817042 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:28.819485 (+ 2443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:28.830084 (+ 10599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:28.830099 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:28.830731 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:28.831619 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:28.831646 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:28.833456 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:28.833462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:28.835543 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:28.835559 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:28.836683 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:28.836691 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:28.837002 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:28.837845 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:28.837876 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:28.840721 (+ 2845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:28.844752 (+ 4031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:28.844761 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:28.844780 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:28.845377 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:28.845880 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:28.845886 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:28.845889 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:28.846120 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:28.846816 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:28.846824 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:28.847270 (+ 446us) client_negotiation.cc:769] Sending connection context
0619 06:47:28.847592 (+ 322us) client_negotiation.cc:240] Negotiation successful
0619 06:47:28.848259 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1511,"thread_start_us":1212,"threads_started":1}
I20260619 06:47:28.851725 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:28.809655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:46429 (local address 127.16.79.126:41619)
0619 06:47:28.829249 (+ 19594us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:28.829257 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:28.829283 (+ 26us) server_negotiation.cc:407] Connection header received
0619 06:47:28.829358 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:28.829364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:28.829451 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:28.831853 (+ 2402us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:28.832032 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:28.833251 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:28.835742 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:28.836422 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:28.837463 (+ 1041us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:28.841005 (+ 3542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:28.841013 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:28.841017 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:28.841057 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:28.844592 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:28.845555 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:28.845560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:28.845563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:28.845734 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:28.846269 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:28.846273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:28.846277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:28.846645 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:28.850967 (+ 4322us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:28.851117 (+ 150us) server_negotiation.cc:299] Negotiation successful
0619 06:47:28.851354 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19431,"thread_start_us":19924,"threads_started":1}
I20260619 06:47:28.853715 29892 heartbeater.cc:347] Connected to a master server at 127.16.79.126:41619
I20260619 06:47:28.854067 29892 heartbeater.cc:464] Registering TS with master...
I20260619 06:47:28.854969 29892 heartbeater.cc:511] Master 127.16.79.126:41619 requested a full tablet report, sending...
I20260619 06:47:28.857230 29551 ts_manager.cc:194] Registered new tserver with Master: cb44a34eb6404b358c47cffa27fd40a7 (127.16.79.66:45003)
I20260619 06:47:28.859411 29551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:46429
Jun 19 06:47:28 dist-test-slave-5w2x krb5kdc[28540](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781851648, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:29.218312 29748 heartbeater.cc:464] Registering TS with master...
I20260619 06:47:29.218739 29748 heartbeater.cc:511] Master 127.16.79.126:41619 requested a full tablet report, sending...
I20260619 06:47:29.219893 29551 ts_manager.cc:194] Registered new tserver with Master: 1077e6bd463c401cbe13d84c1bd16294 (127.16.79.65:34367)
I20260619 06:47:29.220894 29551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:39617
W20260619 06:47:29.285347 29896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:29.285784 29896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:29.285998 29896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:29.297420 29896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:29.297719 29896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:29.297937 29896 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:47:29.313341 29896 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.16.79.84:40663/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:37445
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=46179
--tserver_master_addrs=127.16.79.126:41619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:29.315129 29896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:29.316856 29896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:29.338867 29902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:29.340821 29903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:29.350736 29896 server_base.cc:1034] running on GCE node
W20260619 06:47:29.351593 29905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:29.352603 29896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:29.354094 29896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:29.355396 29896 hybrid_clock.cc:648] HybridClock initialized: now 1781851649355345 us; error 46 us; skew 500 ppm
I20260619 06:47:29.355840 29896 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:29 dist-test-slave-5w2x krb5kdc[28540](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851649, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:29.360457 29896 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:47:29.380405 29896 webserver.cc:460] Webserver started at http://127.16.79.67:46179/ using document root <none> and password file <none>
I20260619 06:47:29.381325 29896 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:29.381520 29896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 06:47:29 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851648, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
Jun 19 06:47:29 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851649, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
I20260619 06:47:29.414304 29896 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.005s sys 0.000s
I20260619 06:47:29.417618 29913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:29.419493 29896 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260619 06:47:29.419816 29896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "0b092762c65f4be7aba5015481182e66"
format_stamp: "Formatted at 2026-06-19 06:47:26 on dist-test-slave-5w2x"
server_key: "5e523732075e4934f5a88e42df1370bd"
server_key_iv: "c31c26cf414c9f2a147ff54a0d05ec1c"
server_key_version: "kuduclusterkey@0"
I20260619 06:47:29.421577 29896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:29.515226 29896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:29.520665 29896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:29.521216 29896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:29.522562 29896 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:47:29.524462 29896 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:47:29.524613 29896 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:29.524758 29896 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:47:29.524857 29896 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:29.668457 29896 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:37445
I20260619 06:47:29.668543 30025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:37445 every 8 connection(s)
I20260619 06:47:29.670266 29896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:47:29.673612 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 29896
Jun 19 06:47:29 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851649, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:29.713882 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:29.676566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:56831 (local address 127.16.79.126:41619)
0619 06:47:29.688649 (+ 12083us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:29.688653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:29.688670 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:47:29.688743 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:29.688748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:29.688867 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:29.688982 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:29.691384 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.692087 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.696011 (+ 3924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.696672 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.696775 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:29.702670 (+ 5895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:29.702678 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:29.702682 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:29.702724 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:29.706253 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:29.707259 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:29.707266 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:29.707270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:29.707423 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:29.709645 (+ 2222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:29.709654 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:29.709660 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:29.710056 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:29.710177 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:29.713205 (+ 3028us) server_negotiation.cc:299] Negotiation successful
0619 06:47:29.713437 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11935,"spinlock_wait_cycles":1408,"thread_start_us":11872,"threads_started":1}
I20260619 06:47:29.717233 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:29.676250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46200 (local address 127.16.79.126:41619)
0619 06:47:29.680497 (+ 4247us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:29.680503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:29.688574 (+ 8071us) server_negotiation.cc:407] Connection header received
0619 06:47:29.688665 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:29.688671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:29.688807 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:29.688950 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:29.693328 (+ 4378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.694507 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.696983 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.697598 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.697715 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:29.707907 (+ 10192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:29.707912 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:29.707916 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:29.707950 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:29.711620 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:29.712524 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:29.712529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:29.712532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:29.712665 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:29.714251 (+ 1586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:29.714256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:29.714259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:29.714603 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:29.714896 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:29.716601 (+ 1705us) server_negotiation.cc:299] Negotiation successful
0619 06:47:29.716843 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4112,"thread_start_us":3981,"threads_started":1}
I20260619 06:47:29.718991 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:29.675466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:41619 (local address 127.16.79.67:56831)
0619 06:47:29.678333 (+ 2867us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:29.678393 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:29.680774 (+ 2381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:29.689508 (+ 8734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:29.689523 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:29.690202 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:29.691113 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:29.691146 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.692280 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.692288 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:29.695803 (+ 3515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:29.695846 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.698084 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.698092 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:29.698392 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:29.699563 (+ 1171us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:29.699595 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:29.702396 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:29.706445 (+ 4049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:29.706456 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:29.706475 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:29.707110 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:29.707582 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:29.707586 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:29.707589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:29.708664 (+ 1075us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:29.711887 (+ 3223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:29.711896 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:29.712398 (+ 502us) client_negotiation.cc:769] Sending connection context
0619 06:47:29.715212 (+ 2814us) client_negotiation.cc:240] Negotiation successful
0619 06:47:29.715810 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2040,"thread_start_us":4038,"threads_started":1}
I20260619 06:47:29.720912 30026 heartbeater.cc:347] Connected to a master server at 127.16.79.126:41619
I20260619 06:47:29.721290 30026 heartbeater.cc:464] Registering TS with master...
I20260619 06:47:29.722205 30026 heartbeater.cc:511] Master 127.16.79.126:41619 requested a full tablet report, sending...
I20260619 06:47:29.723870 29551 ts_manager.cc:194] Registered new tserver with Master: 0b092762c65f4be7aba5015481182e66 (127.16.79.67:37445)
I20260619 06:47:29.725009 29551 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:56831
I20260619 06:47:29.733314 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 06:47:29 dist-test-slave-5w2x krb5kdc[28540](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851648, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for HTTP/127.16.79.84@KRBTEST.COM
I20260619 06:47:29.759205 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:29.737000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46206 (local address 127.16.79.126:41619)
0619 06:47:29.737187 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:29.737193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:29.740736 (+ 3543us) server_negotiation.cc:407] Connection header received
0619 06:47:29.740933 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:29.740938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:29.741038 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:29.741156 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:29.743156 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.744272 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.746407 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.747054 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.747154 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:29.752349 (+ 5195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:29.752356 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:29.752361 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:29.752407 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:29.755748 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:29.756680 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:29.756686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:29.756690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:29.756886 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:29.757378 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:29.757384 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:29.757387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:29.757780 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:29.758016 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:29.758616 (+ 600us) server_negotiation.cc:299] Negotiation successful
0619 06:47:29.758855 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 06:47:29.774104 29551 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:47:29.787196 29551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:47:29.843814 30044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:29.817642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45832 (local address 127.16.79.65:34367)
0619 06:47:29.832833 (+ 15191us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:29.832843 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:29.834633 (+ 1790us) server_negotiation.cc:407] Connection header received
0619 06:47:29.834984 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:29.834993 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:29.835273 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:29.835529 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:47:29.836587 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.838028 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.840772 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.842076 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.842177 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:29.842435 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:29.843245 (+ 810us) server_negotiation.cc:299] Negotiation successful
0619 06:47:29.843461 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14992,"thread_start_us":15208,"threads_started":1}
I20260619 06:47:29.844985 30047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:29.816945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:34367 (local address 127.0.0.1:45832)
0619 06:47:29.834351 (+ 17406us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:29.834397 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:29.834724 (+ 327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:29.835847 (+ 1123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:29.835856 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:29.835890 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:47:29.836352 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:29.836368 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.838215 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.838220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:29.840581 (+ 2361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:29.840602 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.842647 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.842653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:29.842877 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:29.842994 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 06:47:29.844399 (+ 1405us) client_negotiation.cc:240] Negotiation successful
0619 06:47:29.844606 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17127,"mutex_wait_us":23,"thread_start_us":24427,"threads_started":1}
I20260619 06:47:29.849530 29682 tablet_service.cc:1467] Processing CreateTablet for tablet 57901035368642d6910d5ab3bed3bf57 (DEFAULT_TABLE table=test-table [id=bcdcd170ae91424bbb2d2c7ebbc91fdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:47:29.851207 29682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57901035368642d6910d5ab3bed3bf57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:47:29.852610 30048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:29.833116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45446 (local address 127.16.79.67:37445)
0619 06:47:29.833855 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:29.833867 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:29.841849 (+ 7982us) server_negotiation.cc:407] Connection header received
0619 06:47:29.841974 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:29.841982 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:29.842261 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:29.842465 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:47:29.843455 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.845060 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.847559 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.849001 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.849118 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:29.849345 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:29.852058 (+ 2713us) server_negotiation.cc:299] Negotiation successful
0619 06:47:29.852257 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":386,"threads_started":1}
I20260619 06:47:29.853406 30043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:29.832769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:37445 (local address 127.0.0.1:45446)
0619 06:47:29.841652 (+ 8883us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:29.841687 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:29.841838 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:29.842786 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:29.842793 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:29.842833 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:47:29.843267 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:29.843279 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.845244 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.845251 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:29.847371 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:29.847387 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.849550 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.849559 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:29.849724 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:29.849756 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 06:47:29.852881 (+ 3125us) client_negotiation.cc:240] Negotiation successful
0619 06:47:29.853030 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8741,"mutex_wait_us":26,"thread_start_us":1164,"threads_started":1}
I20260619 06:47:29.857542 29961 tablet_service.cc:1467] Processing CreateTablet for tablet 57901035368642d6910d5ab3bed3bf57 (DEFAULT_TABLE table=test-table [id=bcdcd170ae91424bbb2d2c7ebbc91fdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:47:29.859882 30045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:29.819721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:45003 (local address 127.0.0.1:34376)
0619 06:47:29.840948 (+ 21227us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:29.840975 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:29.841124 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:29.845375 (+ 4251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:29.845383 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:29.845424 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:47:29.850039 (+ 4615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:29.850054 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.854607 (+ 4553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.854613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:29.856777 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:29.856793 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.859032 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.859040 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:29.859227 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:29.859264 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 06:47:29.859352 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 06:47:29.859487 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21091,"mutex_wait_us":4151,"thread_start_us":20951,"threads_started":1}
I20260619 06:47:29.860628 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:29.820068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34376 (local address 127.16.79.66:45003)
0619 06:47:29.844320 (+ 24252us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:29.844332 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:29.844370 (+ 38us) server_negotiation.cc:407] Connection header received
0619 06:47:29.844498 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:29.844509 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:29.844816 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:29.845053 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:47:29.850281 (+ 5228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.851834 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.856988 (+ 5154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.858479 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.858617 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:29.858863 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:29.860118 (+ 1255us) server_negotiation.cc:299] Negotiation successful
0619 06:47:29.860316 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23992,"thread_start_us":25648,"threads_started":1}
I20260619 06:47:29.860872 29961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57901035368642d6910d5ab3bed3bf57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:47:29.864027 29892 heartbeater.cc:503] Master 127.16.79.126:41619 was elected leader, sending a full tablet report...
I20260619 06:47:29.864511 29825 tablet_service.cc:1467] Processing CreateTablet for tablet 57901035368642d6910d5ab3bed3bf57 (DEFAULT_TABLE table=test-table [id=bcdcd170ae91424bbb2d2c7ebbc91fdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:47:29.866052 29825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57901035368642d6910d5ab3bed3bf57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:47:29.886912 30049 tablet_bootstrap.cc:492] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7: Bootstrap starting.
I20260619 06:47:29.889026 30050 tablet_bootstrap.cc:492] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66: Bootstrap starting.
I20260619 06:47:29.891496 30049 tablet_bootstrap.cc:654] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7: Neither blocks nor log segments found. Creating new log.
I20260619 06:47:29.892891 30049 log.cc:824] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:29.895098 30050 tablet_bootstrap.cc:654] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66: Neither blocks nor log segments found. Creating new log.
I20260619 06:47:29.896543 30050 log.cc:824] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:29.902616 30049 tablet_bootstrap.cc:492] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7: No bootstrap required, opened a new log
I20260619 06:47:29.903066 30049 ts_tablet_manager.cc:1397] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.010s
I20260619 06:47:29.910287 30051 tablet_bootstrap.cc:492] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294: Bootstrap starting.
I20260619 06:47:29.914400 30050 tablet_bootstrap.cc:492] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66: No bootstrap required, opened a new log
I20260619 06:47:29.914781 30050 ts_tablet_manager.cc:1397] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66: Time spent bootstrapping tablet: real 0.026s user 0.016s sys 0.005s
I20260619 06:47:29.915156 30049 raft_consensus.cc:348] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1077e6bd463c401cbe13d84c1bd16294" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 34367 } } peers { permanent_uuid: "cb44a34eb6404b358c47cffa27fd40a7" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45003 } } peers { permanent_uuid: "0b092762c65f4be7aba5015481182e66" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 37445 } }
I20260619 06:47:29.915845 30051 tablet_bootstrap.cc:654] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294: Neither blocks nor log segments found. Creating new log.
I20260619 06:47:29.915830 30049 raft_consensus.cc:374] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:47:29.915940 30049 raft_consensus.cc:729] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb44a34eb6404b358c47cffa27fd40a7, State: Initialized, Role: FOLLOWER
I20260619 06:47:29.916958 30049 consensus_queue.cc:260] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1077e6bd463c401cbe13d84c1bd16294" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 34367 } } peers { permanent_uuid: "cb44a34eb6404b358c47cffa27fd40a7" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45003 } } peers { permanent_uuid: "0b092762c65f4be7aba5015481182e66" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 37445 } }
I20260619 06:47:29.920017 30051 log.cc:824] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:29.925091 30049 ts_tablet_manager.cc:1428] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7: Time spent starting tablet: real 0.022s user 0.014s sys 0.004s
I20260619 06:47:29.930882 30050 raft_consensus.cc:348] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1077e6bd463c401cbe13d84c1bd16294" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 34367 } } peers { permanent_uuid: "cb44a34eb6404b358c47cffa27fd40a7" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45003 } } peers { permanent_uuid: "0b092762c65f4be7aba5015481182e66" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 37445 } }
I20260619 06:47:29.931584 30050 raft_consensus.cc:374] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:47:29.931711 30050 raft_consensus.cc:729] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b092762c65f4be7aba5015481182e66, State: Initialized, Role: FOLLOWER
I20260619 06:47:29.932629 30050 consensus_queue.cc:260] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1077e6bd463c401cbe13d84c1bd16294" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 34367 } } peers { permanent_uuid: "cb44a34eb6404b358c47cffa27fd40a7" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45003 } } peers { permanent_uuid: "0b092762c65f4be7aba5015481182e66" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 37445 } }
I20260619 06:47:29.934085 30026 heartbeater.cc:503] Master 127.16.79.126:41619 was elected leader, sending a full tablet report...
I20260619 06:47:29.934950 30050 ts_tablet_manager.cc:1428] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66: Time spent starting tablet: real 0.020s user 0.001s sys 0.012s
I20260619 06:47:29.936753 30051 tablet_bootstrap.cc:492] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294: No bootstrap required, opened a new log
I20260619 06:47:29.937112 30051 ts_tablet_manager.cc:1397] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.007s
I20260619 06:47:29.947463 30051 raft_consensus.cc:348] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1077e6bd463c401cbe13d84c1bd16294" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 34367 } } peers { permanent_uuid: "cb44a34eb6404b358c47cffa27fd40a7" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45003 } } peers { permanent_uuid: "0b092762c65f4be7aba5015481182e66" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 37445 } }
I20260619 06:47:29.948042 30051 raft_consensus.cc:374] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:47:29.948196 30051 raft_consensus.cc:729] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1077e6bd463c401cbe13d84c1bd16294, State: Initialized, Role: FOLLOWER
I20260619 06:47:29.948943 30051 consensus_queue.cc:260] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1077e6bd463c401cbe13d84c1bd16294" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 34367 } } peers { permanent_uuid: "cb44a34eb6404b358c47cffa27fd40a7" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45003 } } peers { permanent_uuid: "0b092762c65f4be7aba5015481182e66" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 37445 } }
I20260619 06:47:29.951736 30056 raft_consensus.cc:482] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:47:29.952111 30056 raft_consensus.cc:504] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1077e6bd463c401cbe13d84c1bd16294" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 34367 } } peers { permanent_uuid: "cb44a34eb6404b358c47cffa27fd40a7" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45003 } } peers { permanent_uuid: "0b092762c65f4be7aba5015481182e66" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 37445 } }
I20260619 06:47:29.955776 30056 leader_election.cc:288] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1077e6bd463c401cbe13d84c1bd16294 (127.16.79.65:34367), cb44a34eb6404b358c47cffa27fd40a7 (127.16.79.66:45003)
I20260619 06:47:29.962806 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:29.954963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:34367 (local address 127.16.79.67:49697)
0619 06:47:29.955428 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:29.955449 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:29.955568 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:29.956348 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:29.956355 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:29.956387 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:47:29.956661 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:29.956676 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.958241 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.958246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:29.960280 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:29.960298 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.962041 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.962047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:29.962236 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:29.962271 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 06:47:29.962384 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 06:47:29.962527 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353}
I20260619 06:47:29.963629 30044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:29.955505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:49697 (local address 127.16.79.65:34367)
0619 06:47:29.955696 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:29.955704 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:29.955728 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:47:29.955814 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:29.955819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:29.955895 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:29.956006 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:47:29.956868 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.958066 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.960491 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.961701 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.961869 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:29.961946 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:29.963245 (+ 1299us) server_negotiation.cc:299] Negotiation successful
0619 06:47:29.963395 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:47:29.965004 29748 heartbeater.cc:503] Master 127.16.79.126:41619 was elected leader, sending a full tablet report...
I20260619 06:47:29.965066 29702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57901035368642d6910d5ab3bed3bf57" candidate_uuid: "0b092762c65f4be7aba5015481182e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1077e6bd463c401cbe13d84c1bd16294" is_pre_election: true
I20260619 06:47:29.965757 29702 raft_consensus.cc:2454] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b092762c65f4be7aba5015481182e66 in term 0.
I20260619 06:47:29.966536 29915 leader_election.cc:302] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b092762c65f4be7aba5015481182e66, 1077e6bd463c401cbe13d84c1bd16294; no voters:
I20260619 06:47:29.967013 30056 raft_consensus.cc:2781] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:47:29.967156 30056 raft_consensus.cc:482] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:47:29.967242 30056 raft_consensus.cc:3037] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:47:29.969512 30051 ts_tablet_manager.cc:1428] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294: Time spent starting tablet: real 0.032s user 0.017s sys 0.001s
I20260619 06:47:29.970546 30056 raft_consensus.cc:504] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1077e6bd463c401cbe13d84c1bd16294" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 34367 } } peers { permanent_uuid: "cb44a34eb6404b358c47cffa27fd40a7" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45003 } } peers { permanent_uuid: "0b092762c65f4be7aba5015481182e66" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 37445 } }
I20260619 06:47:29.971565 30056 leader_election.cc:288] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [CANDIDATE]: Term 1 election: Requested vote from peers 1077e6bd463c401cbe13d84c1bd16294 (127.16.79.65:34367), cb44a34eb6404b358c47cffa27fd40a7 (127.16.79.66:45003)
I20260619 06:47:29.971899 29702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57901035368642d6910d5ab3bed3bf57" candidate_uuid: "0b092762c65f4be7aba5015481182e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1077e6bd463c401cbe13d84c1bd16294"
I20260619 06:47:29.972046 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:29.955154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:60973 (local address 127.16.79.66:45003)
0619 06:47:29.955383 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:29.955389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:29.965218 (+ 9829us) server_negotiation.cc:407] Connection header received
0619 06:47:29.965600 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:29.965606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:29.965703 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:29.965803 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:47:29.966537 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.967364 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.969487 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.970831 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.971022 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:29.971096 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:29.971659 (+ 563us) server_negotiation.cc:299] Negotiation successful
0619 06:47:29.971801 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 06:47:29.972111 29702 raft_consensus.cc:3037] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:47:29.972811 30058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:29.955161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:45003 (local address 127.16.79.67:60973)
0619 06:47:29.965084 (+ 9923us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:29.965111 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:29.965404 (+ 293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:29.965842 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:29.965847 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:29.965876 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:47:29.966287 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:29.966300 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.967540 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.967543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:29.969306 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:29.969322 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:29.971259 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:29.971266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:29.971453 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:29.971488 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 06:47:29.972336 (+ 848us) client_negotiation.cc:240] Negotiation successful
0619 06:47:29.972481 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9734,"thread_start_us":2257,"threads_started":1}
I20260619 06:47:29.973965 29844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57901035368642d6910d5ab3bed3bf57" candidate_uuid: "0b092762c65f4be7aba5015481182e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb44a34eb6404b358c47cffa27fd40a7"
I20260619 06:47:29.974009 29845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57901035368642d6910d5ab3bed3bf57" candidate_uuid: "0b092762c65f4be7aba5015481182e66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb44a34eb6404b358c47cffa27fd40a7" is_pre_election: true
I20260619 06:47:29.974514 29844 raft_consensus.cc:3037] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:47:29.975093 29702 raft_consensus.cc:2454] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b092762c65f4be7aba5015481182e66 in term 1.
I20260619 06:47:29.975912 29915 leader_election.cc:302] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b092762c65f4be7aba5015481182e66, 1077e6bd463c401cbe13d84c1bd16294; no voters:
I20260619 06:47:29.976229 30056 raft_consensus.cc:2781] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:47:29.976758 30056 raft_consensus.cc:686] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [term 1 LEADER]: Becoming Leader. State: Replica: 0b092762c65f4be7aba5015481182e66, State: Running, Role: LEADER
I20260619 06:47:29.977299 30056 consensus_queue.cc:237] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1077e6bd463c401cbe13d84c1bd16294" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 34367 } } peers { permanent_uuid: "cb44a34eb6404b358c47cffa27fd40a7" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45003 } } peers { permanent_uuid: "0b092762c65f4be7aba5015481182e66" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 37445 } }
I20260619 06:47:29.977895 29844 raft_consensus.cc:2454] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b092762c65f4be7aba5015481182e66 in term 1.
I20260619 06:47:29.982501 29550 catalog_manager.cc:5495] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b092762c65f4be7aba5015481182e66 (127.16.79.67). New cstate: current_term: 1 leader_uuid: "0b092762c65f4be7aba5015481182e66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1077e6bd463c401cbe13d84c1bd16294" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 34367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb44a34eb6404b358c47cffa27fd40a7" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b092762c65f4be7aba5015481182e66" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 37445 } health_report { overall_health: HEALTHY } } }
I20260619 06:47:30.030179 30048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:30.022989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45458 (local address 127.16.79.67:37445)
0619 06:47:30.023206 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:30.023212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:30.023236 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:47:30.023530 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:30.023538 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:30.023638 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:30.023809 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:47:30.024349 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:30.025442 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:30.027502 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:30.028110 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:30.028258 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:30.028839 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:47:30.029533 (+ 694us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:47:30.029692 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:30.029811 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 06:47:30.029898 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 06:47:30.040803 29844 raft_consensus.cc:1261] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7 [term 1 FOLLOWER]: Refusing update from remote peer 0b092762c65f4be7aba5015481182e66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:47:30.041941 30056 consensus_queue.cc:1035] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb44a34eb6404b358c47cffa27fd40a7" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 45003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:47:30.048920 29702 raft_consensus.cc:1261] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294 [term 1 FOLLOWER]: Refusing update from remote peer 0b092762c65f4be7aba5015481182e66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:47:30.050046 30056 consensus_queue.cc:1035] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1077e6bd463c401cbe13d84c1bd16294" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 34367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 06:47:30.062884 29893 tablet.cc:2369] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:47:30.073429 30062 mvcc.cc:205] Tried to move back new op lower bound from 7298464358553911296 to 7298464358311632896. Current Snapshot: MvccSnapshot[applied={T|T < 7298464358553911296}]
I20260619 06:47:30.073902 30064 mvcc.cc:205] Tried to move back new op lower bound from 7298464358553911296 to 7298464358311632896. Current Snapshot: MvccSnapshot[applied={T|T < 7298464358553911296}]
I20260619 06:47:30.096714 30067 mvcc.cc:205] Tried to move back new op lower bound from 7298464358553911296 to 7298464358311632896. Current Snapshot: MvccSnapshot[applied={T|T < 7298464358553911296 or (T in {7298464358553911296})}]
I20260619 06:47:30.108477 29550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:47:30.108791 29550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:47:30.113188 29550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 142cb03bca934dce8390b18652704813: Sending DeleteTablet for 3 replicas of tablet 57901035368642d6910d5ab3bed3bf57
I20260619 06:47:30.114516 29682 tablet_service.cc:1514] Processing DeleteTablet for tablet 57901035368642d6910d5ab3bed3bf57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:47:30 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:45832
I20260619 06:47:30.115449 30080 tablet_replica.cc:330] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294: stopping tablet replica
I20260619 06:47:30.115988 30080 raft_consensus.cc:2229] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:47:30.116647 30080 raft_consensus.cc:2258] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:47:30.117206 29825 tablet_service.cc:1514] Processing DeleteTablet for tablet 57901035368642d6910d5ab3bed3bf57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:47:30 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:34376
I20260619 06:47:30.119246 30080 ts_tablet_manager.cc:1905] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:47:30.120083 29961 tablet_service.cc:1514] Processing DeleteTablet for tablet 57901035368642d6910d5ab3bed3bf57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:47:30 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:45446
I20260619 06:47:30.126603 30080 ts_tablet_manager.cc:1918] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:47:30.126973 30080 log.cc:1196] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/57901035368642d6910d5ab3bed3bf57
I20260619 06:47:30.127722 30080 ts_tablet_manager.cc:1939] T 57901035368642d6910d5ab3bed3bf57 P 1077e6bd463c401cbe13d84c1bd16294: Deleting consensus metadata
I20260619 06:47:30.129858 29536 catalog_manager.cc:4841] TS 1077e6bd463c401cbe13d84c1bd16294 (127.16.79.65:34367): tablet 57901035368642d6910d5ab3bed3bf57 (table test-table [id=bcdcd170ae91424bbb2d2c7ebbc91fdf]) successfully deleted
I20260619 06:47:30.130141 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 29588
I20260619 06:47:30.140404 30082 tablet_replica.cc:330] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66: stopping tablet replica
I20260619 06:47:30.140930 30082 raft_consensus.cc:2229] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:47:30.141604 30082 raft_consensus.cc:2258] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:47:30.142427 30081 tablet_replica.cc:330] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7: stopping tablet replica
I20260619 06:47:30.142853 30081 raft_consensus.cc:2229] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:47:30.143399 30081 raft_consensus.cc:2258] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:47:30.145603 30082 ts_tablet_manager.cc:1905] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:47:30.148578 30081 ts_tablet_manager.cc:1905] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:47:30.155520 30081 ts_tablet_manager.cc:1918] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:47:30.155702 30081 log.cc:1196] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/57901035368642d6910d5ab3bed3bf57
I20260619 06:47:30.156262 30081 ts_tablet_manager.cc:1939] T 57901035368642d6910d5ab3bed3bf57 P cb44a34eb6404b358c47cffa27fd40a7: Deleting consensus metadata
I20260619 06:47:30.156971 30044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:30.131892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45844 (local address 127.16.79.65:34367)
0619 06:47:30.132120 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:30.132127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:30.132166 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:47:30.132234 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:30.132238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:30.132349 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:30.132456 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:30.135246 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:30.136124 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:30.137945 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:30.138554 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:30.138675 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:30.148224 (+ 9549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:30.148245 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:30.148263 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:30.148322 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:30.152458 (+ 4136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:30.153473 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:30.153483 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:30.153487 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:30.153650 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:30.154123 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:30.154129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:30.154132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:30.154632 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:30.154935 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:30.156352 (+ 1417us) server_negotiation.cc:299] Negotiation successful
0619 06:47:30.156590 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:47:30.157675 29743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:30.161616 29535 catalog_manager.cc:4841] TS cb44a34eb6404b358c47cffa27fd40a7 (127.16.79.66:45003): tablet 57901035368642d6910d5ab3bed3bf57 (table test-table [id=bcdcd170ae91424bbb2d2c7ebbc91fdf]) successfully deleted
I20260619 06:47:30.163058 30082 ts_tablet_manager.cc:1918] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:47:30.163246 30082 log.cc:1196] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/57901035368642d6910d5ab3bed3bf57
I20260619 06:47:30.163798 30082 ts_tablet_manager.cc:1939] T 57901035368642d6910d5ab3bed3bf57 P 0b092762c65f4be7aba5015481182e66: Deleting consensus metadata
I20260619 06:47:30.165998 29537 catalog_manager.cc:4841] TS 0b092762c65f4be7aba5015481182e66 (127.16.79.67:37445): tablet 57901035368642d6910d5ab3bed3bf57 (table test-table [id=bcdcd170ae91424bbb2d2c7ebbc91fdf]) successfully deleted
I20260619 06:47:30.253280 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 29588
I20260619 06:47:30.271149 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 29752
I20260619 06:47:30.285100 30046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:30.271957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34384 (local address 127.16.79.66:45003)
0619 06:47:30.272158 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:30.272165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:30.272317 (+ 152us) server_negotiation.cc:407] Connection header received
0619 06:47:30.272530 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:30.272534 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:30.272599 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:30.272708 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:30.273992 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:30.274720 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:30.275967 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:30.276444 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:30.276582 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:30.279121 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:30.279136 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:30.279150 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:30.279189 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:30.282239 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:30.282937 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:30.282945 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:30.282947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:30.283075 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:30.283455 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:30.283461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:30.283464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:30.283824 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:30.284130 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:30.284485 (+ 355us) server_negotiation.cc:299] Negotiation successful
0619 06:47:30.284706 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:47:30.285811 29885 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:30.377303 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 29752
I20260619 06:47:30.394750 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 29896
I20260619 06:47:30.409583 30048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:30.395846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45468 (local address 127.16.79.67:37445)
0619 06:47:30.396033 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:30.396040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:30.396064 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:47:30.396124 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:30.396128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:30.396263 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:30.396397 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:30.397697 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:30.398562 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:30.399894 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:30.400540 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:30.400685 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:30.403242 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:30.403265 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:30.403281 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:30.403330 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:30.406587 (+ 3257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:30.407279 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:30.407288 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:30.407291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:30.407456 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:30.407852 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:30.407859 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:30.407862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:30.408263 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:30.408569 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:30.408861 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 06:47:30.409076 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 06:47:30.410218 30021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:30.540673 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 29896
I20260619 06:47:30.561714 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 29518
I20260619 06:47:30.562988 29581 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:30.695375 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 29518
I20260619 06:47:30.721005 16701 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 06:47:31.056731 16701 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 06:47:31.056926 16701 mini_ranger.cc:66] Stopping Ranger...
I20260619 06:47:31.402338 16701 mini_ranger.cc:68] Stopped Ranger
2026-06-19 06:47:31.402 UTC [28570] LOG: received smart shutdown request
2026-06-19 06:47:31.405 UTC [28570] LOG: background worker "logical replication launcher" (PID 28579) exited with exit code 1
2026-06-19 06:47:31.405 UTC [28574] LOG: shutting down
2026-06-19 06:47:31.566 UTC [28570] LOG: database system is shut down
2026-06-19T06:47:31Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53086 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:47:31 dist-test-slave-5w2x krb5kdc[30095](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:47:31 dist-test-slave-5w2x krb5kdc[30095](info): set up 2 sockets
Jun 19 06:47:31 dist-test-slave-5w2x krb5kdc[30095](info): commencing operation
krb5kdc: starting...
W20260619 06:47:32.824636 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.010s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:47:32 dist-test-slave-5w2x krb5kdc[30095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:47:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:47:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:47:33.053690 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:34329
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:34329,127.16.79.125:36727,127.16.79.124:42897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:47:33.391824 30108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:33.392199 30108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:33.392396 30108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:33.399806 30108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:47:33.399919 30108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:33.400032 30108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:33.400136 30108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:47:33.400244 30108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:47:33.410360 30108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:34329,127.16.79.125:36727,127.16.79.124:42897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:34329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:33.411836 30108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:33.413228 30108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:33.425102 30113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:33.425069 30114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:33.425668 30116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:33.426192 30108 server_base.cc:1034] running on GCE node
I20260619 06:47:33.427037 30108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:33.428536 30108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:33.429791 30108 hybrid_clock.cc:648] HybridClock initialized: now 1781851653429750 us; error 41 us; skew 500 ppm
I20260619 06:47:33.430166 30108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:33 dist-test-slave-5w2x krb5kdc[30095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851653, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:33.433461 30108 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:47:33.434957 30108 webserver.cc:460] Webserver started at http://127.16.79.126:38429/ using document root <none> and password file <none>
I20260619 06:47:33.435652 30108 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:33.435758 30108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:33.436010 30108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:47:33.439020 30108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "14caeeb0647a468abed65c9209c89c83"
format_stamp: "Formatted at 2026-06-19 06:47:33 on dist-test-slave-5w2x"
server_key: "6f600c163f34d7b17000a9625bd38606"
server_key_iv: "9cb9dd5991804f56ccd13484c34ce643"
server_key_version: "encryptionkey@0"
I20260619 06:47:33.439810 30108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "14caeeb0647a468abed65c9209c89c83"
format_stamp: "Formatted at 2026-06-19 06:47:33 on dist-test-slave-5w2x"
server_key: "6f600c163f34d7b17000a9625bd38606"
server_key_iv: "9cb9dd5991804f56ccd13484c34ce643"
server_key_version: "encryptionkey@0"
I20260619 06:47:33.444892 30108 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:47:33.448405 30123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:33.449754 30108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:47:33.449972 30108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "14caeeb0647a468abed65c9209c89c83"
format_stamp: "Formatted at 2026-06-19 06:47:33 on dist-test-slave-5w2x"
server_key: "6f600c163f34d7b17000a9625bd38606"
server_key_iv: "9cb9dd5991804f56ccd13484c34ce643"
server_key_version: "encryptionkey@0"
I20260619 06:47:33.450186 30108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:33.466814 30108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:33.471227 30108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:33.471531 30108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:33.490911 30108 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:34329
I20260619 06:47:33.490947 30174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:34329 every 8 connection(s)
I20260619 06:47:33.492512 30108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:47:33.495728 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 30108
I20260619 06:47:33.495967 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:47:33.496826 16701 external_mini_cluster.cc:1428] Setting key 454a263c151efd9b5a2a834871f9ac2c
I20260619 06:47:33.496907 30175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:47:33.506552 30175 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.16.79.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.125@KRBTEST.COM" created.
Jun 19 06:47:33 dist-test-slave-5w2x krb5kdc[30095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851653, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:33.530884 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:33.510033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:34329 (local address 127.0.0.1:52294)
0619 06:47:33.510867 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:33.510917 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:33.513021 (+ 2104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:33.513867 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:33.513874 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:33.514570 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:33.514898 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:33.514915 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:33.516584 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:33.516594 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:33.517665 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:33.517676 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:33.518577 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:33.518584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:33.518767 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:33.519449 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:33.519471 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:33.521861 (+ 2390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:33.526596 (+ 4735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:33.526607 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:33.526616 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:33.527023 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:33.527511 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:33.527514 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:33.527516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:33.527671 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:33.528697 (+ 1026us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:33.528708 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:33.528975 (+ 267us) client_negotiation.cc:769] Sending connection context
0619 06:47:33.529298 (+ 323us) client_negotiation.cc:240] Negotiation successful
0619 06:47:33.529922 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":62,"thread_start_us":359,"threads_started":1}
I20260619 06:47:33.530946 30179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:33.511442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52294 (local address 127.16.79.126:34329)
0619 06:47:33.512188 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:33.512196 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:33.512245 (+ 49us) server_negotiation.cc:407] Connection header received
0619 06:47:33.513399 (+ 1154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:33.513404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:33.513637 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:33.513842 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:33.515110 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:33.516425 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:33.517803 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:33.518344 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:33.518540 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:33.522127 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:33.522141 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:33.522161 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:33.522207 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:33.526421 (+ 4214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:33.527163 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:33.527175 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:33.527181 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:33.527387 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:33.527784 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:33.527792 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:33.527798 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:33.528552 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:33.528713 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:33.529426 (+ 713us) server_negotiation.cc:299] Negotiation successful
0619 06:47:33.529705 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":324,"threads_started":1}
I20260619 06:47:33.534204 30175 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:47:33.535435 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:33.534910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:36727 (local address 127.0.0.1:44312)
0619 06:47:33.535118 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:33.535255 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:36727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260619 06:47:33.535743 30124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.79.125:36727: connect: Connection refused (error 111)
I20260619 06:47:33.537539 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:33.537040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:36727 (local address 127.0.0.1:44322)
0619 06:47:33.537235 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:33.537346 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:36727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260619 06:47:33.537760 30175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.125:36727: Network error: Client connection negotiation failed: client connection to 127.16.79.125:36727: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.125.keytab.
Entry for principal kudu/127.16.79.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.125.keytab.
I20260619 06:47:33.549510 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.125:36727
--webserver_interface=127.16.79.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.125.keytab
--principal=kudu/127.16.79.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:34329,127.16.79.125:36727,127.16.79.124:42897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:47:33.587193 30175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } attempt: 1
I20260619 06:47:33.588524 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:33.587945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:36727 (local address 127.0.0.1:44336)
0619 06:47:33.588195 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:33.588363 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:36727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260619 06:47:33.589570 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:33.589170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:36727 (local address 127.0.0.1:44344)
0619 06:47:33.589324 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:33.589409 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:36727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 06:47:33.589763 30175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.125:36727: Network error: Client connection negotiation failed: client connection to 127.16.79.125:36727: connect: Connection refused (error 111)
I20260619 06:47:33.658026 30175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } attempt: 2
I20260619 06:47:33.659337 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:33.658722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:36727 (local address 127.0.0.1:44350)
0619 06:47:33.659014 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:33.659147 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:36727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 06:47:33.660496 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:33.659964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:36727 (local address 127.0.0.1:44362)
0619 06:47:33.660192 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:33.660307 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:36727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 06:47:33.660765 30175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.125:36727: Network error: Client connection negotiation failed: client connection to 127.16.79.125:36727: connect: Connection refused (error 111)
I20260619 06:47:33.752125 30175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } attempt: 3
I20260619 06:47:33.753575 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:33.752984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:36727 (local address 127.0.0.1:44372)
0619 06:47:33.753246 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:33.753400 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:36727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260619 06:47:33.754720 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:33.754323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:36727 (local address 127.0.0.1:44376)
0619 06:47:33.754468 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:33.754551 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:36727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 06:47:33.754990 30175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.125:36727: Network error: Client connection negotiation failed: client connection to 127.16.79.125:36727: connect: Connection refused (error 111)
W20260619 06:47:33.891731 30182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:33.892019 30182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:33.892227 30182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 06:47:33.898239 30175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } attempt: 4
I20260619 06:47:33.899462 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:33.898884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:36727 (local address 127.0.0.1:44392)
0619 06:47:33.899190 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:33.899295 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:36727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260619 06:47:33.900058 30182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:47:33.900261 30182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:33.900360 30182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:33.900446 30182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:47:33.900513 30182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:47:33.900532 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:33.900079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:36727 (local address 127.0.0.1:44404)
0619 06:47:33.900245 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:33.900369 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:36727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 06:47:33.900732 30175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.125:36727: Network error: Client connection negotiation failed: client connection to 127.16.79.125:36727: connect: Connection refused (error 111)
I20260619 06:47:33.910651 30182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:34329,127.16.79.125:36727,127.16.79.124:42897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.125.keytab
--principal=kudu/127.16.79.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.125:36727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.79.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:33.911998 30182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:33.913312 30182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:33.924978 30190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:33.925024 30188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:33.924942 30187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:33.929563 30182 server_base.cc:1034] running on GCE node
I20260619 06:47:33.930357 30182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:33.931798 30182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:33.933038 30182 hybrid_clock.cc:648] HybridClock initialized: now 1781851653932991 us; error 45 us; skew 500 ppm
I20260619 06:47:33.933406 30182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:33 dist-test-slave-5w2x krb5kdc[30095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851653, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:33.936617 30182 init.cc:377] Logged in from keytab as kudu/127.16.79.125@KRBTEST.COM (short username kudu)
I20260619 06:47:33.938138 30182 webserver.cc:460] Webserver started at http://127.16.79.125:37401/ using document root <none> and password file <none>
I20260619 06:47:33.938858 30182 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:33.939004 30182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:33.939309 30182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:47:33.942409 30182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/data/instance:
uuid: "9176d92406ad4b9583fa6723682abd6c"
format_stamp: "Formatted at 2026-06-19 06:47:33 on dist-test-slave-5w2x"
server_key: "017e652a6d27e8609b4ca7faf8811eb7"
server_key_iv: "90c45e775bb0527e8a07da22013facb0"
server_key_version: "encryptionkey@0"
I20260619 06:47:33.943233 30182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/wal/instance:
uuid: "9176d92406ad4b9583fa6723682abd6c"
format_stamp: "Formatted at 2026-06-19 06:47:33 on dist-test-slave-5w2x"
server_key: "017e652a6d27e8609b4ca7faf8811eb7"
server_key_iv: "90c45e775bb0527e8a07da22013facb0"
server_key_version: "encryptionkey@0"
I20260619 06:47:33.948608 30182 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 06:47:33.952087 30197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:33.953675 30182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:47:33.953886 30182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/wal
uuid: "9176d92406ad4b9583fa6723682abd6c"
format_stamp: "Formatted at 2026-06-19 06:47:33 on dist-test-slave-5w2x"
server_key: "017e652a6d27e8609b4ca7faf8811eb7"
server_key_iv: "90c45e775bb0527e8a07da22013facb0"
server_key_version: "encryptionkey@0"
I20260619 06:47:33.954087 30182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:33.975137 30182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:33.979408 30182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:33.979717 30182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:33.998270 30182 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.125:36727
I20260619 06:47:33.998319 30248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.125:36727 every 8 connection(s)
I20260619 06:47:33.999732 30182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/data/info.pb
I20260619 06:47:34.002244 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 30182
I20260619 06:47:34.002465 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-1/wal/instance
I20260619 06:47:34.003006 16701 external_mini_cluster.cc:1428] Setting key 2b544f00470dc24ab1668dd0d2ab349d
I20260619 06:47:34.004521 30249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:47:34.013827 30249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.16.79.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.124@KRBTEST.COM" created.
Jun 19 06:47:34 dist-test-slave-5w2x krb5kdc[30095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851653, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.125@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:34.034780 30179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.016653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52306 (local address 127.16.79.126:34329)
0619 06:47:34.016846 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:34.016851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:34.018274 (+ 1423us) server_negotiation.cc:407] Connection header received
0619 06:47:34.020180 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:34.020186 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:34.020282 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:34.020393 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:34.022017 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.022709 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.024308 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.024768 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.024912 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.028738 (+ 3826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:34.028745 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:34.028756 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:34.028792 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:34.031524 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.032391 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.032398 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.032402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.032533 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.032980 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.032985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.032988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.033292 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:34.033458 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:34.034204 (+ 746us) server_negotiation.cc:299] Negotiation successful
0619 06:47:34.034451 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:47:34.035818 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.017276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:34329 (local address 127.0.0.1:52306)
0619 06:47:34.018111 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.018148 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:34.019886 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:34.020629 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:34.020637 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:34.021258 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:34.021829 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:34.021846 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.022876 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.022881 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:34.024112 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:34.024124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.024999 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.025005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:34.025232 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.025952 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:34.025974 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:34.028488 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:34.031698 (+ 3210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:34.031709 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:34.031729 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:34.032269 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:34.032659 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:34.032664 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:34.032668 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:34.032856 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:34.033447 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:34.033456 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:34.033824 (+ 368us) client_negotiation.cc:769] Sending connection context
0619 06:47:34.034106 (+ 282us) client_negotiation.cc:240] Negotiation successful
0619 06:47:34.034659 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":370,"threads_started":1}
I20260619 06:47:34.037773 30249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:47:34 dist-test-slave-5w2x krb5kdc[30095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851653, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.125@KRBTEST.COM for kudu/127.16.79.125@KRBTEST.COM
I20260619 06:47:34.052665 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.038614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:36727 (local address 127.0.0.1:44408)
0619 06:47:34.038802 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.038820 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:34.038931 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:34.039945 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:34.039949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:34.040460 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:34.040582 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:34.040590 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.041932 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.041940 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:34.042778 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:34.042789 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.043608 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.043613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:34.043860 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.044516 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:34.044540 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:34.046564 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:34.050190 (+ 3626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:34.050194 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:34.050197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:34.050610 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:34.051035 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:34.051038 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:34.051041 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:34.051172 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:34.051827 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:34.051831 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:34.051974 (+ 143us) client_negotiation.cc:769] Sending connection context
0619 06:47:34.052096 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 06:47:34.052382 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":45}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.124.keytab.
Entry for principal kudu/127.16.79.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.124.keytab.
I20260619 06:47:34.053082 30254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.038894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44408 (local address 127.16.79.125:36727)
0619 06:47:34.039530 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:34.039536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:34.039561 (+ 25us) server_negotiation.cc:407] Connection header received
0619 06:47:34.039604 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:34.039609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:34.039798 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:34.039956 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:34.040729 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.041791 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.042918 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.043457 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.043597 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.046776 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:34.046787 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:34.046801 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:34.046832 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:34.050048 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.050726 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.050733 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.050735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.050889 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.051271 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.051274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.051276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.051705 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:34.051949 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:34.052296 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 06:47:34.052483 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":494,"thread_start_us":354,"threads_started":1}
I20260619 06:47:34.054224 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.124:42897
--webserver_interface=127.16.79.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.124.keytab
--principal=kudu/127.16.79.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:34329,127.16.79.125:36727,127.16.79.124:42897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:47:34.055500 30249 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:47:34.056890 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.056314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52658)
0619 06:47:34.056541 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.056689 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260619 06:47:34.057190 30200 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
I20260619 06:47:34.059056 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.058622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52662)
0619 06:47:34.058809 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.058909 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 06:47:34.059267 30249 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:42897: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
I20260619 06:47:34.108618 30249 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } attempt: 1
I20260619 06:47:34.109872 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.109402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52670)
0619 06:47:34.109595 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.109697 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260619 06:47:34.110885 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.110545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52674)
0619 06:47:34.110715 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.110783 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 06:47:34.111091 30249 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:42897: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
I20260619 06:47:34.179313 30249 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } attempt: 2
I20260619 06:47:34.180572 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.180096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52676)
0619 06:47:34.180321 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.180415 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260619 06:47:34.181649 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.181239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52692)
0619 06:47:34.181423 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.181508 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260619 06:47:34.181926 30249 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:42897: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
I20260619 06:47:34.200014 30175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } attempt: 5
Jun 19 06:47:34 dist-test-slave-5w2x krb5kdc[30095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851653, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for kudu/127.16.79.125@KRBTEST.COM
I20260619 06:47:34.214612 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.200839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:36727 (local address 127.0.0.1:44414)
0619 06:47:34.201069 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.201092 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:34.201250 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:34.201636 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:34.201641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:34.202061 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:34.202248 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:34.202257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.203248 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.203260 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:34.204131 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:34.204207 (+ 76us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.204935 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.204941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:34.205147 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.205805 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:34.205824 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:34.208082 (+ 2258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:34.212062 (+ 3980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:34.212066 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:34.212069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:34.212522 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:34.213000 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:34.213018 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:34.213021 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:34.213162 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:34.213861 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:34.213864 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:34.214023 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 06:47:34.214141 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 06:47:34.214362 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 06:47:34.214828 30254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.200936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.16.79.125:36727)
0619 06:47:34.201132 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:34.201140 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:34.201164 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:47:34.201407 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:34.201411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:34.201494 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:34.201601 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:34.202416 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.203103 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.204332 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.204780 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.204919 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.208332 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:34.208339 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:34.208344 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:34.208386 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:34.211906 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.212646 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.212653 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.212659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.212845 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.213300 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.213308 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.213312 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.213717 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:34.213871 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:34.214290 (+ 419us) server_negotiation.cc:299] Negotiation successful
0619 06:47:34.214525 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260619 06:47:34.215965 30175 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:47:34.217239 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.216751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52698)
0619 06:47:34.216964 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.217065 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260619 06:47:34.218309 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.217966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52710)
0619 06:47:34.218122 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.218202 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 06:47:34.218515 30175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:42897: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
I20260619 06:47:34.269764 30175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } attempt: 1
I20260619 06:47:34.270952 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.270448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52726)
0619 06:47:34.270663 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.270765 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260619 06:47:34.272025 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.271670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52730)
0619 06:47:34.271847 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.271920 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260619 06:47:34.272295 30175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:42897: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
I20260619 06:47:34.273140 30249 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } attempt: 3
I20260619 06:47:34.274166 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.273650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52742)
0619 06:47:34.273879 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.273998 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260619 06:47:34.275400 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.274930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52746)
0619 06:47:34.275166 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.275258 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
W20260619 06:47:34.275655 30249 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:42897: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
I20260619 06:47:34.340523 30175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } attempt: 2
I20260619 06:47:34.341732 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.341228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52756)
0619 06:47:34.341462 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.341550 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260619 06:47:34.342880 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.342454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52758)
0619 06:47:34.342635 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.342765 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260619 06:47:34.343096 30175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:42897: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
W20260619 06:47:34.392222 30256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:34.392590 30256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:34.392776 30256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:34.400151 30256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:47:34.400332 30256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:34.400444 30256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:34.400543 30256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:47:34.400628 30256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:47:34.410687 30256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:34329,127.16.79.125:36727,127.16.79.124:42897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.124.keytab
--principal=kudu/127.16.79.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.124:42897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.79.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:34.412158 30256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:34.413584 30256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 06:47:34.418973 30249 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } attempt: 4
I20260619 06:47:34.420370 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.419647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52774)
0619 06:47:34.419992 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.420111 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260619 06:47:34.421414 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.421126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52788)
0619 06:47:34.421258 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.421320 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 06:47:34.421612 30249 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:42897: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
W20260619 06:47:34.426517 30261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:34.426517 30262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:34.426539 30264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:34.427455 30256 server_base.cc:1034] running on GCE node
I20260619 06:47:34.428297 30256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:34.429768 30256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:34.431020 30256 hybrid_clock.cc:648] HybridClock initialized: now 1781851654430969 us; error 42 us; skew 500 ppm
I20260619 06:47:34.431432 30256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:34 dist-test-slave-5w2x krb5kdc[30095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851654, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:34.434610 30256 init.cc:377] Logged in from keytab as kudu/127.16.79.124@KRBTEST.COM (short username kudu)
I20260619 06:47:34.436106 30256 webserver.cc:460] Webserver started at http://127.16.79.124:41113/ using document root <none> and password file <none>
I20260619 06:47:34.436834 30256 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:34.436933 30256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:34.437170 30256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:47:34.440064 30256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/data/instance:
uuid: "f07c1d29135249f9b2183ec7134bc778"
format_stamp: "Formatted at 2026-06-19 06:47:34 on dist-test-slave-5w2x"
server_key: "6cf6450946f3bc17823bddd4bba42948"
server_key_iv: "b915446a879b3eda630ed20b57857b94"
server_key_version: "encryptionkey@0"
I20260619 06:47:34.440899 30256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/wal/instance:
uuid: "f07c1d29135249f9b2183ec7134bc778"
format_stamp: "Formatted at 2026-06-19 06:47:34 on dist-test-slave-5w2x"
server_key: "6cf6450946f3bc17823bddd4bba42948"
server_key_iv: "b915446a879b3eda630ed20b57857b94"
server_key_version: "encryptionkey@0"
I20260619 06:47:34.445932 30256 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:47:34.449326 30175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } attempt: 3
I20260619 06:47:34.449507 30271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:34.450565 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.450007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52796)
0619 06:47:34.450291 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.450409 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 06:47:34.451135 30256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:47:34.451318 30256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/wal
uuid: "f07c1d29135249f9b2183ec7134bc778"
format_stamp: "Formatted at 2026-06-19 06:47:34 on dist-test-slave-5w2x"
server_key: "6cf6450946f3bc17823bddd4bba42948"
server_key_iv: "b915446a879b3eda630ed20b57857b94"
server_key_version: "encryptionkey@0"
I20260619 06:47:34.451574 30256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:34.451817 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.451317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:52802)
0619 06:47:34.451571 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.451667 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260619 06:47:34.452052 30175 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:42897: Network error: Client connection negotiation failed: client connection to 127.16.79.124:42897: connect: Connection refused (error 111)
I20260619 06:47:34.478492 30256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:34.482704 30256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:34.482998 30256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:34.502218 30256 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.124:42897
I20260619 06:47:34.502259 30322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.124:42897 every 8 connection(s)
I20260619 06:47:34.503530 30256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/data/info.pb
I20260619 06:47:34.507148 30323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:47:34.507510 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 30256
I20260619 06:47:34.507709 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781851451399607-16701-0/minicluster-data/master-2/wal/instance
I20260619 06:47:34.508267 16701 external_mini_cluster.cc:1428] Setting key 46dc6f236cd9963da811f7fe918e0362
I20260619 06:47:34.516381 30323 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:47:34 dist-test-slave-5w2x krb5kdc[30095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.124@KRBTEST.COM
Jun 19 06:47:34 dist-test-slave-5w2x krb5kdc[30095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851654, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.124@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:34.533479 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.510050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51510 (local address 127.16.79.124:42897)
0619 06:47:34.510845 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:34.510855 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:34.510898 (+ 43us) server_negotiation.cc:407] Connection header received
0619 06:47:34.513052 (+ 2154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:34.513074 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:34.513450 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:34.513775 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:34.515597 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.516921 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.519096 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.520286 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.520513 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.525152 (+ 4639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:34.525160 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:34.525171 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:34.525210 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:34.528802 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.529692 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.529698 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.529704 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.529842 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.530228 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.530234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.530238 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.531015 (+ 777us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:34.531360 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:34.531869 (+ 509us) server_negotiation.cc:299] Negotiation successful
0619 06:47:34.532309 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":387,"threads_started":1}
I20260619 06:47:34.535784 30179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.519451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39076 (local address 127.16.79.126:34329)
0619 06:47:34.519635 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:34.519641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:34.520482 (+ 841us) server_negotiation.cc:407] Connection header received
0619 06:47:34.520786 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:34.520791 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:34.520857 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:34.520934 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:34.522475 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.523152 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.524662 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.525291 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.525583 (+ 292us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.529709 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:34.529714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:34.529718 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:34.529751 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:34.532845 (+ 3094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.533730 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.533737 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.533740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.533910 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.534319 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.534323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.534326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.534640 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:34.534800 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:34.535370 (+ 570us) server_negotiation.cc:299] Negotiation successful
0619 06:47:34.535529 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:47:34.535951 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.519417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:34329 (local address 127.0.0.1:39076)
0619 06:47:34.520320 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.520366 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:34.520620 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:34.521038 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:34.521048 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:34.521893 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:34.522292 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:34.522307 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.523312 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.523318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:34.524514 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:34.524526 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.525664 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.525670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:34.525985 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.526597 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:34.526620 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:34.529508 (+ 2888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:34.533000 (+ 3492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:34.533009 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:34.533015 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:34.533580 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:34.534041 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:34.534044 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:34.534047 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:34.534196 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:34.534818 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:34.534826 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:34.535022 (+ 196us) client_negotiation.cc:769] Sending connection context
0619 06:47:34.535254 (+ 232us) client_negotiation.cc:240] Negotiation successful
0619 06:47:34.535561 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":439,"threads_started":1}
I20260619 06:47:34.537909 30323 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:47:34 dist-test-slave-5w2x krb5kdc[30095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851654, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.124@KRBTEST.COM for kudu/127.16.79.125@KRBTEST.COM
Jun 19 06:47:34 dist-test-slave-5w2x krb5kdc[30095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:34.553050 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.538965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:36727 (local address 127.0.0.1:60978)
0619 06:47:34.539322 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.539339 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:34.539429 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:34.539837 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:34.539840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:34.540230 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:34.540370 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:34.540377 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.541314 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.541320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:34.542422 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:34.542437 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.543328 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.543337 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:34.543563 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.544064 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:34.544088 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:34.546607 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:34.550064 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:34.550070 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:34.550075 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:34.550601 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:34.551143 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:34.551146 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:34.551149 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:34.551356 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:34.551996 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:34.552003 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:34.552212 (+ 209us) client_negotiation.cc:769] Sending connection context
0619 06:47:34.552394 (+ 182us) client_negotiation.cc:240] Negotiation successful
0619 06:47:34.552698 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240}
I20260619 06:47:34.553157 30254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.538969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60978 (local address 127.16.79.125:36727)
0619 06:47:34.539175 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:34.539181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:34.539419 (+ 238us) server_negotiation.cc:407] Connection header received
0619 06:47:34.539560 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:34.539566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:34.539642 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:34.539789 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:34.540528 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.541178 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.542576 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.543171 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.543272 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.546797 (+ 3525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:34.546803 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:34.546808 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:34.546852 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:34.549923 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.550741 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.550750 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.550757 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.550967 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.551480 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.551483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.551485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.551876 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:34.552026 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:34.552500 (+ 474us) server_negotiation.cc:299] Negotiation successful
0619 06:47:34.552742 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:47:34.553449 30179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.537174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39082 (local address 127.16.79.126:34329)
0619 06:47:34.537340 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:34.537345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:34.537457 (+ 112us) server_negotiation.cc:407] Connection header received
0619 06:47:34.537554 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:34.537559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:34.537633 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:34.537749 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:34.539365 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.540384 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.541868 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.542274 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.542380 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.547673 (+ 5293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:34.547677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:34.547681 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:34.547711 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:34.550484 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.551337 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.551343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.551346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.551511 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.551926 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.551950 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.551955 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.552334 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:34.552611 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:34.552888 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 06:47:34.553116 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:47:34.554677 30323 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:47:34 dist-test-slave-5w2x krb5kdc[30095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851654, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.124@KRBTEST.COM for kudu/127.16.79.124@KRBTEST.COM
Jun 19 06:47:34 dist-test-slave-5w2x krb5kdc[30095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.125@KRBTEST.COM
I20260619 06:47:34.568113 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.555766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51516 (local address 127.16.79.124:42897)
0619 06:47:34.555916 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:34.555922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:34.555935 (+ 13us) server_negotiation.cc:407] Connection header received
0619 06:47:34.555978 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:34.555981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:34.556026 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:34.556123 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:34.556818 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.557633 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.558837 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.559330 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.559467 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.562860 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:34.562865 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:34.562868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:34.562894 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:34.565532 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.566343 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.566347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.566349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.566462 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.566881 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.566884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.566887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.567158 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:34.567274 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:34.567685 (+ 411us) server_negotiation.cc:299] Negotiation successful
0619 06:47:34.567857 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 06:47:34.568583 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.555388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:51516)
0619 06:47:34.555540 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.555558 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:34.555641 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:34.556158 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:34.556163 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:34.556530 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:34.556655 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:34.556662 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.557808 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.557812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:34.558713 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:34.558723 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.559481 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.559487 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:34.559667 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.560286 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:34.560309 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:34.562674 (+ 2365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:34.565703 (+ 3029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:34.565708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:34.565710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:34.566210 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:34.566577 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:34.566582 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:34.566586 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:34.566763 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:34.567283 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:34.567291 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:34.567453 (+ 162us) client_negotiation.cc:769] Sending connection context
0619 06:47:34.567604 (+ 151us) client_negotiation.cc:240] Negotiation successful
0619 06:47:34.568090 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":119}
I20260619 06:47:34.571336 30254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.555211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60994 (local address 127.16.79.125:36727)
0619 06:47:34.555371 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:34.555377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:34.555481 (+ 104us) server_negotiation.cc:407] Connection header received
0619 06:47:34.555631 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:34.555636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:34.555711 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:34.555829 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:34.557224 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.558213 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.559537 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.560083 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.560251 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.565024 (+ 4773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:34.565028 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:34.565032 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:34.565064 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:34.568373 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.569339 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.569344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.569348 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.569502 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.569909 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.569914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.569918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.570208 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:34.570496 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:34.570770 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 06:47:34.571007 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 06:47:34.573918 30323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778: Bootstrap starting.
I20260619 06:47:34.576292 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:47:34.577608 30323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778: Neither blocks nor log segments found. Creating new log.
I20260619 06:47:34.578666 30323 log.cc:824] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:34.582017 30323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778: No bootstrap required, opened a new log
I20260619 06:47:34.594053 30323 raft_consensus.cc:348] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } }
I20260619 06:47:34.594642 30323 raft_consensus.cc:374] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:47:34.594779 30323 raft_consensus.cc:729] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f07c1d29135249f9b2183ec7134bc778, State: Initialized, Role: FOLLOWER
I20260619 06:47:34.595693 30323 consensus_queue.cc:260] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } }
I20260619 06:47:34.602905 30336 sys_catalog.cc:455] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } } }
I20260619 06:47:34.603399 30336 sys_catalog.cc:458] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:47:34.605212 30179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.581145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39086 (local address 127.16.79.126:34329)
0619 06:47:34.581330 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:34.581338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:34.585447 (+ 4109us) server_negotiation.cc:407] Connection header received
0619 06:47:34.585753 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:34.585761 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:34.585847 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:34.585922 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:34.588513 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.589634 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.592302 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.592990 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.593100 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.597626 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:34.597634 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:34.597639 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:34.597693 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:34.601415 (+ 3722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.602632 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.602638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.602641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.602821 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.603334 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.603340 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.603343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.603700 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:34.603934 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:34.604616 (+ 682us) server_negotiation.cc:299] Negotiation successful
0619 06:47:34.604849 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:47:34.605217 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.581146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51532 (local address 127.16.79.124:42897)
0619 06:47:34.581347 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:34.581353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:34.585358 (+ 4005us) server_negotiation.cc:407] Connection header received
0619 06:47:34.585778 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:34.585783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:34.585851 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:34.585920 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:34.588515 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.589669 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.592307 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.593042 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.593123 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.597299 (+ 4176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:34.597308 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:34.597313 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:34.597357 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:34.601498 (+ 4141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.602631 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.602637 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.602640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.602821 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.603338 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.603343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.603346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.603690 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:34.603941 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:34.604601 (+ 660us) server_negotiation.cc:299] Negotiation successful
0619 06:47:34.604843 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 06:47:34.605408 30254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.581514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32778 (local address 127.16.79.125:36727)
0619 06:47:34.582088 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:34.582096 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:34.584925 (+ 2829us) server_negotiation.cc:407] Connection header received
0619 06:47:34.585445 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:34.585452 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:34.585537 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:34.585612 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:34.587992 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.589162 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.591419 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.592112 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.592243 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.596369 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:34.596379 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:34.596387 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:34.596450 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:34.601222 (+ 4772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.602812 (+ 1590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.602817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.602820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.602996 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.603542 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.603550 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.603552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.603953 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:34.604188 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:34.604840 (+ 652us) server_negotiation.cc:299] Negotiation successful
0619 06:47:34.605090 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407}
I20260619 06:47:34.605527 30323 sys_catalog.cc:564] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 06:47:34.620004 30347 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:47:34.620164 30347 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:47:34.617380 30256 master_runner.cc:186] Error getting master registration for 127.16.79.126:34329: OK, instance_id { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" instance_seqno: 1781851653485275 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 06:47:34.626739 30256 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 06:47:34.629303 30175 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } attempt: 4
Jun 19 06:47:34 dist-test-slave-5w2x krb5kdc[30095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851653, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for kudu/127.16.79.124@KRBTEST.COM
I20260619 06:47:34.645493 30178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.629918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:51540)
0619 06:47:34.630391 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.630413 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:34.630521 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:34.631176 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:34.631182 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:34.631710 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:34.631916 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:34.631931 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.633074 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.633080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:34.634566 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:34.634582 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.635449 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.635455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:34.635654 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.636356 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:34.636380 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:34.638962 (+ 2582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:34.643067 (+ 4105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:34.643071 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:34.643074 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:34.643441 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:34.643883 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:34.643887 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:34.643889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:34.644032 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:34.644742 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:34.644746 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:34.644905 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 06:47:34.645023 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 06:47:34.645228 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342}
I20260619 06:47:34.646013 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.630184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51540 (local address 127.16.79.124:42897)
0619 06:47:34.630732 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:34.630738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:34.630760 (+ 22us) server_negotiation.cc:407] Connection header received
0619 06:47:34.630816 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:34.630821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:34.630890 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:34.630978 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:34.632074 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.632942 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.634778 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.635301 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.635400 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.639182 (+ 3782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:34.639189 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:34.639194 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:34.639233 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:34.642919 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.643560 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.643566 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.643569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.643741 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.644175 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.644180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.644184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.644598 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:34.644748 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:34.645158 (+ 410us) server_negotiation.cc:299] Negotiation successful
0619 06:47:34.645419 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433}
I20260619 06:47:34.651274 30175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83: Bootstrap starting.
I20260619 06:47:34.654749 30175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83: Neither blocks nor log segments found. Creating new log.
I20260619 06:47:34.655978 30175 log.cc:824] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:34.659303 30175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83: No bootstrap required, opened a new log
I20260619 06:47:34.670542 30175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } }
I20260619 06:47:34.670991 30175 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:47:34.671108 30175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14caeeb0647a468abed65c9209c89c83, State: Initialized, Role: FOLLOWER
I20260619 06:47:34.671792 30175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } }
I20260619 06:47:34.672823 30350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } } }
I20260619 06:47:34.673162 30350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:47:34.674258 30175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 06:47:34.684350 30361 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:47:34.684515 30361 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:47:34.686179 30108 master_runner.cc:186] Error getting master registration for 127.16.79.125:36727: OK, instance_id { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" instance_seqno: 1781851653993188 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 06:47:34.693634 30108 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 06:47:34.704351 30336 raft_consensus.cc:482] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:47:34.704663 30336 raft_consensus.cc:504] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } }
I20260619 06:47:34.706266 30336 leader_election.cc:288] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14caeeb0647a468abed65c9209c89c83 (127.16.79.126:34329), 9176d92406ad4b9583fa6723682abd6c (127.16.79.125:36727)
I20260619 06:47:34.707134 30150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f07c1d29135249f9b2183ec7134bc778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14caeeb0647a468abed65c9209c89c83" is_pre_election: true
I20260619 06:47:34.707894 30150 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f07c1d29135249f9b2183ec7134bc778 in term 0.
I20260619 06:47:34.708552 30275 leader_election.cc:302] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14caeeb0647a468abed65c9209c89c83, f07c1d29135249f9b2183ec7134bc778; no voters:
I20260619 06:47:34.709003 30336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:47:34.709167 30336 raft_consensus.cc:482] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:47:34.709277 30336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:47:34.712728 30336 raft_consensus.cc:504] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } }
I20260619 06:47:34.714313 30150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f07c1d29135249f9b2183ec7134bc778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14caeeb0647a468abed65c9209c89c83"
I20260619 06:47:34.714556 30150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:47:34.719502 30150 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f07c1d29135249f9b2183ec7134bc778 in term 1.
I20260619 06:47:34.720464 30275 leader_election.cc:302] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14caeeb0647a468abed65c9209c89c83, f07c1d29135249f9b2183ec7134bc778; no voters:
I20260619 06:47:34.720829 30249 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } attempt: 5
I20260619 06:47:34.721753 30336 leader_election.cc:288] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [CANDIDATE]: Term 1 election: Requested vote from peers 14caeeb0647a468abed65c9209c89c83 (127.16.79.126:34329), 9176d92406ad4b9583fa6723682abd6c (127.16.79.125:36727)
I20260619 06:47:34.721963 30336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:47:34.722114 30336 raft_consensus.cc:686] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [term 1 LEADER]: Becoming Leader. State: Replica: f07c1d29135249f9b2183ec7134bc778, State: Running, Role: LEADER
I20260619 06:47:34.722718 30336 consensus_queue.cc:237] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } }
W20260619 06:47:34.724550 30129 tablet.cc:2369] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:47:34.735144 30277 tablet.cc:2369] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:47:34.738561 30363 sys_catalog.cc:455] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f07c1d29135249f9b2183ec7134bc778. Latest consensus state: current_term: 1 leader_uuid: "f07c1d29135249f9b2183ec7134bc778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } } }
I20260619 06:47:34.739125 30363 sys_catalog.cc:458] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [sys.catalog]: This master's current role is: LEADER
Jun 19 06:47:34 dist-test-slave-5w2x krb5kdc[30095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851653, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.125@KRBTEST.COM for kudu/127.16.79.124@KRBTEST.COM
I20260619 06:47:34.748657 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.721453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:42897 (local address 127.0.0.1:51556)
0619 06:47:34.721833 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:34.721855 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:34.721973 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:34.729733 (+ 7760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:34.729739 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:34.730312 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:34.730544 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:34.730558 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.732229 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.732238 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:34.733900 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:34.733918 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.735127 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.735133 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:34.735396 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.737256 (+ 1860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:34.737294 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:34.740408 (+ 3114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:34.744877 (+ 4469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:34.744882 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:34.744888 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:34.745812 (+ 924us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:34.746268 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:34.746300 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:34.746305 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:34.746493 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:34.747223 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:34.747231 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:34.747520 (+ 289us) client_negotiation.cc:769] Sending connection context
0619 06:47:34.747679 (+ 159us) client_negotiation.cc:240] Negotiation successful
0619 06:47:34.748001 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260619 06:47:34.749166 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:34.729006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51556 (local address 127.16.79.124:42897)
0619 06:47:34.729174 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:34.729180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:34.729208 (+ 28us) server_negotiation.cc:407] Connection header received
0619 06:47:34.729293 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:34.729300 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:34.729385 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:34.729507 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:34.730736 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.732010 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.734084 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:34.734783 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:34.734901 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:34.740685 (+ 5784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:34.740693 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:34.740697 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:34.740809 (+ 112us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:34.744691 (+ 3882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.745972 (+ 1281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.745978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.745982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.746136 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:34.746623 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:34.746628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:34.746632 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:34.747059 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:34.747214 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:34.747916 (+ 702us) server_negotiation.cc:299] Negotiation successful
0619 06:47:34.748468 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 06:47:34.749676 30365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:47:34.755519 30249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c: Bootstrap starting.
I20260619 06:47:34.759977 30249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c: Neither blocks nor log segments found. Creating new log.
I20260619 06:47:34.761379 30249 log.cc:824] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:34.763695 30365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:47:34.773607 30249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c: No bootstrap required, opened a new log
I20260619 06:47:34.789244 30150 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83 [term 1 FOLLOWER]: Refusing update from remote peer f07c1d29135249f9b2183ec7134bc778: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:47:34.790630 30363 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:47:34.792883 30249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } }
I20260619 06:47:34.793427 30249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:47:34.793601 30249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9176d92406ad4b9583fa6723682abd6c, State: Initialized, Role: FOLLOWER
I20260619 06:47:34.794414 30249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } }
I20260619 06:47:34.809629 30368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } } }
I20260619 06:47:34.810087 30368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:47:34.811164 30249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:47:34.811594 30222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:47:34.815682 30223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f07c1d29135249f9b2183ec7134bc778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9176d92406ad4b9583fa6723682abd6c"
I20260619 06:47:34.817665 30224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f07c1d29135249f9b2183ec7134bc778" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9176d92406ad4b9583fa6723682abd6c" is_pre_election: true
I20260619 06:47:34.815357 30222 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [term 1 FOLLOWER]: Refusing update from remote peer f07c1d29135249f9b2183ec7134bc778: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:47:34.819145 30368 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:47:34.819424 30368 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } }
I20260619 06:47:34.819808 30363 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:47:34.825553 30149 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9176d92406ad4b9583fa6723682abd6c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14caeeb0647a468abed65c9209c89c83" is_pre_election: true
I20260619 06:47:34.826346 30298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9176d92406ad4b9583fa6723682abd6c" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f07c1d29135249f9b2183ec7134bc778" is_pre_election: true
I20260619 06:47:34.827594 30200 leader_election.cc:302] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9176d92406ad4b9583fa6723682abd6c; no voters: 14caeeb0647a468abed65c9209c89c83, f07c1d29135249f9b2183ec7134bc778
I20260619 06:47:34.828318 30368 leader_election.cc:288] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 14caeeb0647a468abed65c9209c89c83 (127.16.79.126:34329), f07c1d29135249f9b2183ec7134bc778 (127.16.79.124:42897)
I20260619 06:47:34.849440 30368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f07c1d29135249f9b2183ec7134bc778. Latest consensus state: current_term: 1 leader_uuid: "f07c1d29135249f9b2183ec7134bc778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } } }
I20260619 06:47:34.849856 30368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:47:34.850198 30373 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:47:34.857683 30367 mvcc.cc:205] Tried to move back new op lower bound from 7298464377979727872 to 7298464377757999104. Current Snapshot: MvccSnapshot[applied={T|T < 7298464377979727872}]
I20260619 06:47:34.854233 30350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f07c1d29135249f9b2183ec7134bc778. Latest consensus state: current_term: 1 leader_uuid: "f07c1d29135249f9b2183ec7134bc778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } } }
I20260619 06:47:34.859386 30350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:47:34.864626 30336 sys_catalog.cc:455] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f07c1d29135249f9b2183ec7134bc778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } } }
I20260619 06:47:34.865077 30336 sys_catalog.cc:458] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [sys.catalog]: This master's current role is: LEADER
I20260619 06:47:34.867107 30365 catalog_manager.cc:1347] Generated new cluster ID: 3440cd53278b45d7a2695e6deaf04109
I20260619 06:47:34.867205 30365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:47:34.868947 30363 sys_catalog.cc:455] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f07c1d29135249f9b2183ec7134bc778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } } }
I20260619 06:47:34.869786 30363 sys_catalog.cc:458] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778 [sys.catalog]: This master's current role is: LEADER
I20260619 06:47:34.877084 30350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f07c1d29135249f9b2183ec7134bc778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } } }
I20260619 06:47:34.877439 30350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:47:34.883790 30373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f07c1d29135249f9b2183ec7134bc778" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14caeeb0647a468abed65c9209c89c83" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34329 } } peers { permanent_uuid: "9176d92406ad4b9583fa6723682abd6c" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 36727 } } peers { permanent_uuid: "f07c1d29135249f9b2183ec7134bc778" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 42897 } } }
I20260619 06:47:34.884367 30373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:47:34.917069 30389 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 06:47:34.924535 30365 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:47:34.926440 30365 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:47:34.941825 30365 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f07c1d29135249f9b2183ec7134bc778: Generated new TSK 0
I20260619 06:47:34.942993 30365 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:47:35.686559 30361 catalog_manager.cc:1259] Loaded cluster ID: 3440cd53278b45d7a2695e6deaf04109
I20260619 06:47:35.686707 30361 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83: loading cluster ID for follower catalog manager: success
I20260619 06:47:35.689458 30361 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83: acquiring CA information for follower catalog manager: success
I20260619 06:47:35.691293 30361 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 14caeeb0647a468abed65c9209c89c83: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 06:47:35.922400 30389 catalog_manager.cc:1259] Loaded cluster ID: 3440cd53278b45d7a2695e6deaf04109
I20260619 06:47:35.922510 30389 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c: loading cluster ID for follower catalog manager: success
I20260619 06:47:35.925271 30389 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c: acquiring CA information for follower catalog manager: success
I20260619 06:47:35.927222 30389 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9176d92406ad4b9583fa6723682abd6c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 06:47:36.032393 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 30108
I20260619 06:47:36.033737 30170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:36.168937 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 30108
I20260619 06:47:36.190157 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 30182
I20260619 06:47:36.191419 30244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:36.303009 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 30182
I20260619 06:47:36.322585 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 30256
I20260619 06:47:36.323719 30318 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:36.431559 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 30256
2026-06-19T06:47:36Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4733 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:47:36 dist-test-slave-5w2x krb5kdc[30400](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:47:36 dist-test-slave-5w2x krb5kdc[30400](info): set up 2 sockets
Jun 19 06:47:36 dist-test-slave-5w2x krb5kdc[30400](info): commencing operation
krb5kdc: starting...
W20260619 06:47:37.536854 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.006s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:47:37 dist-test-slave-5w2x krb5kdc[30400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:47:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:47:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:47:37.771410 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:33053
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:36615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:33053
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:47:38.108587 30413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:38.108879 30413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:38.109056 30413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:38.116605 30413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:47:38.116771 30413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:38.116863 30413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:38.116962 30413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:47:38.117046 30413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:47:38.127121 30413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:36615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:33053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:33053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:38.128538 30413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:38.129904 30413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:38.141552 30418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:38.141589 30419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:38.142241 30421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:38.142923 30413 server_base.cc:1034] running on GCE node
I20260619 06:47:38.143680 30413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:38.145119 30413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:38.146335 30413 hybrid_clock.cc:648] HybridClock initialized: now 1781851658146296 us; error 36 us; skew 500 ppm
I20260619 06:47:38.146695 30413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:38 dist-test-slave-5w2x krb5kdc[30400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851658, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:38.149784 30413 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:47:38.151191 30413 webserver.cc:460] Webserver started at http://127.16.79.126:45809/ using document root <none> and password file <none>
I20260619 06:47:38.151883 30413 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:38.152033 30413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:38.152400 30413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:47:38.155402 30413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "a85dac52e6d946a5a6f61e4a0c842c20"
format_stamp: "Formatted at 2026-06-19 06:47:38 on dist-test-slave-5w2x"
server_key: "05ee9ebd1e42ca617afb09418bbec6c2"
server_key_iv: "7fc53e20291b59974937b6dae4851c0f"
server_key_version: "encryptionkey@0"
I20260619 06:47:38.156270 30413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "a85dac52e6d946a5a6f61e4a0c842c20"
format_stamp: "Formatted at 2026-06-19 06:47:38 on dist-test-slave-5w2x"
server_key: "05ee9ebd1e42ca617afb09418bbec6c2"
server_key_iv: "7fc53e20291b59974937b6dae4851c0f"
server_key_version: "encryptionkey@0"
I20260619 06:47:38.161477 30413 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 06:47:38.165061 30428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:38.166407 30413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:47:38.166600 30413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "a85dac52e6d946a5a6f61e4a0c842c20"
format_stamp: "Formatted at 2026-06-19 06:47:38 on dist-test-slave-5w2x"
server_key: "05ee9ebd1e42ca617afb09418bbec6c2"
server_key_iv: "7fc53e20291b59974937b6dae4851c0f"
server_key_version: "encryptionkey@0"
I20260619 06:47:38.166819 30413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:38.192397 30413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:38.196478 30413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:38.196786 30413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:38.215313 30413 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:33053
I20260619 06:47:38.215360 30479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:33053 every 8 connection(s)
I20260619 06:47:38.216816 30413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:47:38.220588 30480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:47:38.225767 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 30413
I20260619 06:47:38.226009 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:47:38.226838 16701 external_mini_cluster.cc:1428] Setting key 2fc4b4973468e04b50d1236ba194ece8
I20260619 06:47:38.232245 30480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20: Bootstrap starting.
I20260619 06:47:38.235986 30480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20: Neither blocks nor log segments found. Creating new log.
I20260619 06:47:38.237315 30480 log.cc:824] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:38.240953 30480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20: No bootstrap required, opened a new log
Jun 19 06:47:38 dist-test-slave-5w2x krb5kdc[30400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:38.249357 30480 raft_consensus.cc:348] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85dac52e6d946a5a6f61e4a0c842c20" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 33053 } }
I20260619 06:47:38.249778 30480 raft_consensus.cc:374] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:47:38.249914 30480 raft_consensus.cc:729] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a85dac52e6d946a5a6f61e4a0c842c20, State: Initialized, Role: FOLLOWER
I20260619 06:47:38.250602 30480 consensus_queue.cc:260] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85dac52e6d946a5a6f61e4a0c842c20" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 33053 } }
I20260619 06:47:38.250882 30480 raft_consensus.cc:388] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:47:38.251039 30480 raft_consensus.cc:482] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:47:38.251247 30480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:47:38.251437 30483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:38.228759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35188 (local address 127.16.79.126:33053)
0619 06:47:38.229600 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:38.229611 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:38.229662 (+ 51us) server_negotiation.cc:407] Connection header received
0619 06:47:38.231350 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:38.231372 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:38.231745 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:38.232066 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:38.233963 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:38.235264 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:38.237264 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:38.238273 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:38.238503 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:38.243628 (+ 5125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:38.243636 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:38.243650 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:38.243694 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:38.246919 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:38.247708 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:38.247714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:38.247721 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:38.247908 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:38.248417 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:38.248421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:38.248424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:38.248963 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:38.249274 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:38.249774 (+ 500us) server_negotiation.cc:299] Negotiation successful
0619 06:47:38.250236 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":414,"threads_started":1}
I20260619 06:47:38.253765 30480 raft_consensus.cc:504] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85dac52e6d946a5a6f61e4a0c842c20" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 33053 } }
I20260619 06:47:38.254343 30480 leader_election.cc:302] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a85dac52e6d946a5a6f61e4a0c842c20; no voters:
I20260619 06:47:38.255122 30480 leader_election.cc:288] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:47:38.255213 30485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:47:38.255419 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:47:38.257802 30480 sys_catalog.cc:564] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:47:38.262149 30485 raft_consensus.cc:686] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [term 1 LEADER]: Becoming Leader. State: Replica: a85dac52e6d946a5a6f61e4a0c842c20, State: Running, Role: LEADER
I20260619 06:47:38.262993 30485 consensus_queue.cc:237] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85dac52e6d946a5a6f61e4a0c842c20" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 33053 } }
I20260619 06:47:38.266108 30487 sys_catalog.cc:455] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a85dac52e6d946a5a6f61e4a0c842c20. Latest consensus state: current_term: 1 leader_uuid: "a85dac52e6d946a5a6f61e4a0c842c20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85dac52e6d946a5a6f61e4a0c842c20" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 33053 } } }
I20260619 06:47:38.266209 30486 sys_catalog.cc:455] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a85dac52e6d946a5a6f61e4a0c842c20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a85dac52e6d946a5a6f61e4a0c842c20" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 33053 } } }
I20260619 06:47:38.266597 30486 sys_catalog.cc:458] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [sys.catalog]: This master's current role is: LEADER
I20260619 06:47:38.266603 30487 sys_catalog.cc:458] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20 [sys.catalog]: This master's current role is: LEADER
W20260619 06:47:38.272819 30499 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:47:38.272945 30499 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:47:38.276772 30500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:47:38.279024 30500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:47:38.288275 30500 catalog_manager.cc:1347] Generated new cluster ID: ccaa916e1dd540dbb7fd7a7075aa9b8e
I20260619 06:47:38.288403 30500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:47:38.311923 30500 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:47:38.313951 30500 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:47:38.322132 30500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a85dac52e6d946a5a6f61e4a0c842c20: Generated new TSK 0
I20260619 06:47:38.323231 30500 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:47:38.368710 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 30413
I20260619 06:47:38.369951 30475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:38.468050 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 30413
2026-06-19T06:47:38Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2032 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:47:38 dist-test-slave-5w2x krb5kdc[30508](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:47:38 dist-test-slave-5w2x krb5kdc[30508](info): set up 2 sockets
Jun 19 06:47:38 dist-test-slave-5w2x krb5kdc[30508](info): commencing operation
krb5kdc: starting...
W20260619 06:47:39.564574 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.003s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:47:39 dist-test-slave-5w2x krb5kdc[30508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:47:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:47:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:47:39.803892 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:46613
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:46613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:47:40.147779 30521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:40.148069 30521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:40.148221 30521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:40.155438 30521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:47:40.155555 30521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:40.155602 30521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:40.155660 30521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:47:40.155704 30521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:47:40.165896 30521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:46613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:46613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:40.167389 30521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:40.168756 30521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:40.180697 30527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:40.180760 30526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:40.180719 30529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:40.181588 30521 server_base.cc:1034] running on GCE node
I20260619 06:47:40.182322 30521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:40.183691 30521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:40.184901 30521 hybrid_clock.cc:648] HybridClock initialized: now 1781851660184851 us; error 40 us; skew 500 ppm
I20260619 06:47:40.185293 30521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:40 dist-test-slave-5w2x krb5kdc[30508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851660, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:40.188525 30521 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:47:40.189987 30521 webserver.cc:460] Webserver started at http://127.16.79.126:35929/ using document root <none> and password file <none>
I20260619 06:47:40.190693 30521 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:40.190848 30521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:40.191156 30521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:47:40.194254 30521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "6c3c2d38c4b74400a2979735183f3ffd"
format_stamp: "Formatted at 2026-06-19 06:47:40 on dist-test-slave-5w2x"
server_key: "e3231c3825d1c856da19060c846606ce"
server_key_iv: "f0e563c9e66bcdf90b230f0b45a3ea11"
server_key_version: "encryptionkey@0"
I20260619 06:47:40.195042 30521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "6c3c2d38c4b74400a2979735183f3ffd"
format_stamp: "Formatted at 2026-06-19 06:47:40 on dist-test-slave-5w2x"
server_key: "e3231c3825d1c856da19060c846606ce"
server_key_iv: "f0e563c9e66bcdf90b230f0b45a3ea11"
server_key_version: "encryptionkey@0"
I20260619 06:47:40.200562 30521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:47:40.204416 30536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:40.205911 30521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:47:40.206121 30521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "6c3c2d38c4b74400a2979735183f3ffd"
format_stamp: "Formatted at 2026-06-19 06:47:40 on dist-test-slave-5w2x"
server_key: "e3231c3825d1c856da19060c846606ce"
server_key_iv: "f0e563c9e66bcdf90b230f0b45a3ea11"
server_key_version: "encryptionkey@0"
I20260619 06:47:40.206387 30521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:40.226706 30521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:40.231033 30521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:40.231349 30521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:40.250910 30521 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:46613
I20260619 06:47:40.250944 30587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:46613 every 8 connection(s)
I20260619 06:47:40.252401 30521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:47:40.256314 30588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:47:40.259856 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 30521
I20260619 06:47:40.260080 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:47:40.260984 16701 external_mini_cluster.cc:1428] Setting key c90936120ffbe27cf0332c26ae4c2ce4
I20260619 06:47:40.269232 30588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd: Bootstrap starting.
I20260619 06:47:40.272377 30588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd: Neither blocks nor log segments found. Creating new log.
I20260619 06:47:40.273417 30588 log.cc:824] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd: Log is configured to *not* fsync() on all Append() calls
Jun 19 06:47:40 dist-test-slave-5w2x krb5kdc[30508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:40.276372 30588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd: No bootstrap required, opened a new log
I20260619 06:47:40.283821 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:40.263180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54022 (local address 127.16.79.126:46613)
0619 06:47:40.263949 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:40.263960 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:40.264001 (+ 41us) server_negotiation.cc:407] Connection header received
0619 06:47:40.265695 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:40.265717 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:40.266183 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:40.266594 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:40.268073 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:40.269488 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:40.270838 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:40.271526 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:40.271728 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:40.275900 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:40.275908 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:40.275922 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:40.275960 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:40.279272 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:40.280051 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:40.280058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:40.280068 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:40.280313 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:40.280728 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:40.280743 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:40.280746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:40.281335 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:40.281691 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:40.282174 (+ 483us) server_negotiation.cc:299] Negotiation successful
0619 06:47:40.282665 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":350,"threads_started":1}
I20260619 06:47:40.284996 30588 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } }
I20260619 06:47:40.285492 30588 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:47:40.285640 30588 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c3c2d38c4b74400a2979735183f3ffd, State: Initialized, Role: FOLLOWER
I20260619 06:47:40.286223 30588 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } }
I20260619 06:47:40.286492 30588 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:47:40.286645 30588 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:47:40.286828 30588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:47:40.289306 30588 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } }
I20260619 06:47:40.289768 30588 leader_election.cc:302] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c3c2d38c4b74400a2979735183f3ffd; no voters:
I20260619 06:47:40.290323 30588 leader_election.cc:288] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:47:40.290475 30593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:47:40.291201 30593 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 1 LEADER]: Becoming Leader. State: Replica: 6c3c2d38c4b74400a2979735183f3ffd, State: Running, Role: LEADER
I20260619 06:47:40.291982 30593 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } }
I20260619 06:47:40.293392 30588 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:47:40.294973 30595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c3c2d38c4b74400a2979735183f3ffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } } }
I20260619 06:47:40.294960 30594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c3c2d38c4b74400a2979735183f3ffd. Latest consensus state: current_term: 1 leader_uuid: "6c3c2d38c4b74400a2979735183f3ffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } } }
I20260619 06:47:40.295385 30595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [sys.catalog]: This master's current role is: LEADER
I20260619 06:47:40.295432 30594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [sys.catalog]: This master's current role is: LEADER
I20260619 06:47:40.297951 30598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:47:40.303324 30598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:47:40.313470 30598 catalog_manager.cc:1347] Generated new cluster ID: ff4eef734e6246b3adabc2f4356d5de6
I20260619 06:47:40.313612 30598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:47:40.324982 30598 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:47:40.326421 30598 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:47:40.343595 30598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd: Generated new TSK 0
I20260619 06:47:40.344599 30598 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:47:40.397457 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:46613
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:47:40.750686 30614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:40.750972 30614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:40.751164 30614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:40.758636 30614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:40.758754 30614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:40.758963 30614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:47:40.769237 30614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:46613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:40.770603 30614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:40.771889 30614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:40.784971 30619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:40.785006 30622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:40.785173 30620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:40.785877 30614 server_base.cc:1034] running on GCE node
I20260619 06:47:40.786470 30614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:40.787356 30614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:40.788550 30614 hybrid_clock.cc:648] HybridClock initialized: now 1781851660788504 us; error 36 us; skew 500 ppm
I20260619 06:47:40.788893 30614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:40 dist-test-slave-5w2x krb5kdc[30508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851660, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:40.792066 30614 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:47:40.793493 30614 webserver.cc:460] Webserver started at http://127.16.79.65:39897/ using document root <none> and password file <none>
I20260619 06:47:40.794379 30614 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:40.794525 30614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:40.794840 30614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:47:40.797849 30614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "7be5258921314224a3abb9bec547f4f2"
format_stamp: "Formatted at 2026-06-19 06:47:40 on dist-test-slave-5w2x"
server_key: "126746a2345cf1b5b6408b53cd6f1562"
server_key_iv: "9bc70c4eb39fab32713840f6e4becc5c"
server_key_version: "encryptionkey@0"
I20260619 06:47:40.798694 30614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "7be5258921314224a3abb9bec547f4f2"
format_stamp: "Formatted at 2026-06-19 06:47:40 on dist-test-slave-5w2x"
server_key: "126746a2345cf1b5b6408b53cd6f1562"
server_key_iv: "9bc70c4eb39fab32713840f6e4becc5c"
server_key_version: "encryptionkey@0"
I20260619 06:47:40.803870 30614 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260619 06:47:40.807652 30629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:40.809243 30614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:47:40.809435 30614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "7be5258921314224a3abb9bec547f4f2"
format_stamp: "Formatted at 2026-06-19 06:47:40 on dist-test-slave-5w2x"
server_key: "126746a2345cf1b5b6408b53cd6f1562"
server_key_iv: "9bc70c4eb39fab32713840f6e4becc5c"
server_key_version: "encryptionkey@0"
I20260619 06:47:40.809656 30614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:40.827404 30614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:40.831889 30614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:40.832216 30614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:40.833197 30614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:47:40.834618 30614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:47:40.834774 30614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:40.834923 30614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:47:40.834997 30614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:40.869549 30614 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:42065
I20260619 06:47:40.869582 30741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:42065 every 8 connection(s)
I20260619 06:47:40.871371 30614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:47:40.871784 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 30614
I20260619 06:47:40.872037 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:47:40.872931 16701 external_mini_cluster.cc:1428] Setting key 384d6c881e76db9f9c6aa179e7453f48
Jun 19 06:47:40 dist-test-slave-5w2x krb5kdc[30508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851660, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:40.898097 30744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:40.875083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:58811 (local address 127.16.79.126:46613)
0619 06:47:40.875527 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:40.875534 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:40.876873 (+ 1339us) server_negotiation.cc:407] Connection header received
0619 06:47:40.879856 (+ 2983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:40.879860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:40.879944 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:40.880040 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:40.882389 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:40.883131 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:40.885058 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:40.885894 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:40.886061 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:40.890004 (+ 3943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:40.890014 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:40.890022 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:40.890074 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:40.894094 (+ 4020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:40.895244 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:40.895253 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:40.895257 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:40.895419 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:40.895895 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:40.895904 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:40.895910 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:40.896385 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:40.896563 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:40.897306 (+ 743us) server_negotiation.cc:299] Negotiation successful
0619 06:47:40.897589 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":5100,"threads_started":1}
I20260619 06:47:40.898857 30745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:40.875760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:58811)
0619 06:47:40.876678 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:40.876738 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:40.879515 (+ 2777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:40.880388 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:40.880404 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:40.881351 (+ 947us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:40.882185 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:40.882223 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:40.883312 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:40.883320 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:40.884883 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:40.884897 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:40.886196 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:40.886201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:40.886449 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:40.887236 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:40.887273 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:40.889698 (+ 2425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:40.894262 (+ 4564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:40.894274 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:40.894290 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:40.895079 (+ 789us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:40.895546 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:40.895549 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:40.895551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:40.895766 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:40.896512 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:40.896518 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:40.896885 (+ 367us) client_negotiation.cc:769] Sending connection context
0619 06:47:40.897122 (+ 237us) client_negotiation.cc:240] Negotiation successful
0619 06:47:40.897529 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":377,"threads_started":1}
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
I20260619 06:47:40.900557 30742 heartbeater.cc:347] Connected to a master server at 127.16.79.126:46613
I20260619 06:47:40.900883 30742 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
I20260619 06:47:40.901803 30742 heartbeater.cc:511] Master 127.16.79.126:46613 requested a full tablet report, sending...
I20260619 06:47:40.904248 30553 ts_manager.cc:194] Registered new tserver with Master: 7be5258921314224a3abb9bec547f4f2 (127.16.79.65:42065)
I20260619 06:47:40.906492 30553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:58811
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:47:40.931488 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:46613
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:47:41.261270 30748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:41.261634 30748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:41.261792 30748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:41.273520 30748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:41.273655 30748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:41.273838 30748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:47:41.284045 30748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:46613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:41.285359 30748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:41.286643 30748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:41.299860 30753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:41.299942 30756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:41.299961 30754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:41.300818 30748 server_base.cc:1034] running on GCE node
I20260619 06:47:41.301433 30748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:41.302373 30748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:41.303619 30748 hybrid_clock.cc:648] HybridClock initialized: now 1781851661303576 us; error 36 us; skew 500 ppm
I20260619 06:47:41.303954 30748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:41 dist-test-slave-5w2x krb5kdc[30508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851661, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:41.307317 30748 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:47:41.308873 30748 webserver.cc:460] Webserver started at http://127.16.79.66:43011/ using document root <none> and password file <none>
I20260619 06:47:41.309520 30748 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:41.309655 30748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:41.309998 30748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:47:41.313184 30748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "d26783cc2ffe498d93143110c417d64b"
format_stamp: "Formatted at 2026-06-19 06:47:41 on dist-test-slave-5w2x"
server_key: "26a07fd5fb6ac478f92645f17db1d4be"
server_key_iv: "6146becce51bf312741260063f6f575e"
server_key_version: "encryptionkey@0"
I20260619 06:47:41.313967 30748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "d26783cc2ffe498d93143110c417d64b"
format_stamp: "Formatted at 2026-06-19 06:47:41 on dist-test-slave-5w2x"
server_key: "26a07fd5fb6ac478f92645f17db1d4be"
server_key_iv: "6146becce51bf312741260063f6f575e"
server_key_version: "encryptionkey@0"
I20260619 06:47:41.319120 30748 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260619 06:47:41.322731 30763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:41.324208 30748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:47:41.324345 30748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "d26783cc2ffe498d93143110c417d64b"
format_stamp: "Formatted at 2026-06-19 06:47:41 on dist-test-slave-5w2x"
server_key: "26a07fd5fb6ac478f92645f17db1d4be"
server_key_iv: "6146becce51bf312741260063f6f575e"
server_key_version: "encryptionkey@0"
I20260619 06:47:41.324512 30748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:41.337749 30748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:41.341643 30748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:41.341892 30748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:41.342748 30748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:47:41.344237 30748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:47:41.344310 30748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:41.344395 30748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:47:41.344439 30748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:41.377149 30748 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:35795
I20260619 06:47:41.377297 30875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:35795 every 8 connection(s)
I20260619 06:47:41.378449 30748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:47:41.384258 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 30748
I20260619 06:47:41.384469 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:47:41.385056 16701 external_mini_cluster.cc:1428] Setting key 0c8a55ffd140ee52d30c6fdb579bfe94
Jun 19 06:47:41 dist-test-slave-5w2x krb5kdc[30508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851661, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:41.399926 30744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:41.380944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:59269 (local address 127.16.79.126:46613)
0619 06:47:41.381120 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:41.381124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:41.382221 (+ 1097us) server_negotiation.cc:407] Connection header received
0619 06:47:41.384038 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:41.384042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:41.384112 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:41.384259 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:41.386136 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.386879 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.388466 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.388894 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.389032 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:41.392999 (+ 3967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:41.393004 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:41.393007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:41.393040 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:41.396622 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:41.397376 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:41.397385 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:41.397391 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:41.397561 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:41.397965 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:41.397972 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:41.397978 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:41.398399 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:41.398553 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:41.399222 (+ 669us) server_negotiation.cc:299] Negotiation successful
0619 06:47:41.399492 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:47:41.400843 30878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:41.381376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:59269)
0619 06:47:41.382055 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:41.382099 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:41.383695 (+ 1596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:41.384514 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:41.384527 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:41.385257 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:41.385955 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:41.385974 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.387044 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.387048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:41.388326 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:41.388338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.389118 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.389125 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:41.389349 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:41.390023 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:41.390045 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:41.392765 (+ 2720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:41.396773 (+ 4008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:41.396779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:41.396792 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:41.397242 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:41.397671 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:41.397674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:41.397677 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:41.397854 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:41.398528 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:41.398534 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:41.398852 (+ 318us) client_negotiation.cc:769] Sending connection context
0619 06:47:41.399087 (+ 235us) client_negotiation.cc:240] Negotiation successful
0619 06:47:41.399603 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":257,"threads_started":1}
I20260619 06:47:41.402348 30876 heartbeater.cc:347] Connected to a master server at 127.16.79.126:46613
I20260619 06:47:41.402676 30876 heartbeater.cc:464] Registering TS with master...
I20260619 06:47:41.403633 30876 heartbeater.cc:511] Master 127.16.79.126:46613 requested a full tablet report, sending...
I20260619 06:47:41.405444 30553 ts_manager.cc:194] Registered new tserver with Master: d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795)
I20260619 06:47:41.406288 30553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:59269
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:47:41.436690 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:46613
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:47:41.768870 30881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:41.769214 30881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:41.769385 30881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:41.777235 30881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:41.777400 30881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:41.777518 30881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:47:41.787739 30881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:46613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:41.789064 30881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:41.790324 30881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:41.803568 30886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:41.803494 30889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:41.803539 30887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:41.804667 30881 server_base.cc:1034] running on GCE node
I20260619 06:47:41.805403 30881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:41.806432 30881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:41.807667 30881 hybrid_clock.cc:648] HybridClock initialized: now 1781851661807605 us; error 52 us; skew 500 ppm
I20260619 06:47:41.808084 30881 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:41 dist-test-slave-5w2x krb5kdc[30508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851661, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:41.816015 30881 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:47:41.817757 30881 webserver.cc:460] Webserver started at http://127.16.79.67:33735/ using document root <none> and password file <none>
I20260619 06:47:41.818456 30881 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:41.818564 30881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:41.818831 30881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:47:41.821889 30881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "e467369eecdd47158114ea2eb8bd20ac"
format_stamp: "Formatted at 2026-06-19 06:47:41 on dist-test-slave-5w2x"
server_key: "f723f5a77a09c70446ea167529533b3f"
server_key_iv: "567d31e6c0758d07d949229bdfd31881"
server_key_version: "encryptionkey@0"
I20260619 06:47:41.822649 30881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "e467369eecdd47158114ea2eb8bd20ac"
format_stamp: "Formatted at 2026-06-19 06:47:41 on dist-test-slave-5w2x"
server_key: "f723f5a77a09c70446ea167529533b3f"
server_key_iv: "567d31e6c0758d07d949229bdfd31881"
server_key_version: "encryptionkey@0"
I20260619 06:47:41.828230 30881 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:47:41.831913 30896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:41.833374 30881 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 06:47:41.833561 30881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "e467369eecdd47158114ea2eb8bd20ac"
format_stamp: "Formatted at 2026-06-19 06:47:41 on dist-test-slave-5w2x"
server_key: "f723f5a77a09c70446ea167529533b3f"
server_key_iv: "567d31e6c0758d07d949229bdfd31881"
server_key_version: "encryptionkey@0"
I20260619 06:47:41.833778 30881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:41.849749 30881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:41.854132 30881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:41.854446 30881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:41.855428 30881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:47:41.856873 30881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:47:41.856981 30881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:41.857136 30881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:47:41.857218 30881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:41.893033 30881 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:44827
I20260619 06:47:41.893085 31008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:44827 every 8 connection(s)
I20260619 06:47:41.894435 30881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:47:41.899768 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 30881
I20260619 06:47:41.899999 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:47:41.900581 16701 external_mini_cluster.cc:1428] Setting key dd09df8d5023ed2e6cc03c5f03791115
Jun 19 06:47:41 dist-test-slave-5w2x krb5kdc[30508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851661, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:41.910163 30742 heartbeater.cc:503] Master 127.16.79.126:46613 was elected leader, sending a full tablet report...
I20260619 06:47:41.917860 30744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:41.897267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:60781 (local address 127.16.79.126:46613)
0619 06:47:41.897457 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:41.897461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:41.898693 (+ 1232us) server_negotiation.cc:407] Connection header received
0619 06:47:41.901237 (+ 2544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:41.901243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:41.901340 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:41.901445 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:41.903741 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.904587 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.906816 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.907230 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.907340 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:41.911260 (+ 3920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:41.911266 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:41.911271 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:41.911308 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:41.914339 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:41.915320 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:41.915325 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:41.915327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:41.915506 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:41.916051 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:41.916059 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:41.916063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:41.916508 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:41.916633 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:41.917449 (+ 816us) server_negotiation.cc:299] Negotiation successful
0619 06:47:41.917610 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 06:47:41.918990 31011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:41.897799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:60781)
0619 06:47:41.898506 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:41.898549 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:41.900864 (+ 2315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:41.901918 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:41.901931 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:41.902687 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:41.903554 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:41.903581 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.904757 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.904761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:41.906647 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:41.906661 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.907456 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.907461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:41.907671 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:41.908471 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:41.908496 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:41.911057 (+ 2561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:41.914495 (+ 3438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:41.914502 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:41.914524 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:41.915186 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:41.915682 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:41.915688 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:41.915694 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:41.915929 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:41.916690 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:41.916697 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:41.917052 (+ 355us) client_negotiation.cc:769] Sending connection context
0619 06:47:41.917369 (+ 317us) client_negotiation.cc:240] Negotiation successful
0619 06:47:41.917783 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":313,"threads_started":1}
I20260619 06:47:41.920451 31009 heartbeater.cc:347] Connected to a master server at 127.16.79.126:46613
I20260619 06:47:41.920759 31009 heartbeater.cc:464] Registering TS with master...
I20260619 06:47:41.921497 31009 heartbeater.cc:511] Master 127.16.79.126:46613 requested a full tablet report, sending...
I20260619 06:47:41.922892 30553 ts_manager.cc:194] Registered new tserver with Master: e467369eecdd47158114ea2eb8bd20ac (127.16.79.67:44827)
I20260619 06:47:41.923703 30553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:60781
I20260619 06:47:41.928740 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:47:41.945736 30744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:41.933054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54026 (local address 127.16.79.126:46613)
0619 06:47:41.933218 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:41.933232 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:41.933855 (+ 623us) server_negotiation.cc:407] Connection header received
0619 06:47:41.933915 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:41.933918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:41.933980 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:41.934056 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:41.935368 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.936093 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.937664 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.938103 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.938193 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:41.940643 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:41.940648 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:41.940651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:41.940684 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:41.943192 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:41.943861 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:41.943866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:41.943868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:41.944005 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:41.944485 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:41.944489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:41.944491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:41.944814 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:41.945013 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:41.945320 (+ 307us) server_negotiation.cc:299] Negotiation successful
0619 06:47:41.945484 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 06:47:41.949816 30553 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:47:41.956888 30553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:47:41.982087 31024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:41.970356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43306 (local address 127.16.79.66:35795)
0619 06:47:41.974316 (+ 3960us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:41.974329 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:41.974363 (+ 34us) server_negotiation.cc:407] Connection header received
0619 06:47:41.974485 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:41.974495 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:41.974798 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:41.975005 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:47:41.975756 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.977223 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.979168 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.980663 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.980783 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:41.980995 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:41.981594 (+ 599us) server_negotiation.cc:299] Negotiation successful
0619 06:47:41.981779 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1994,"thread_start_us":1880,"threads_started":1}
I20260619 06:47:41.982106 31023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:41.969773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:35795 (local address 127.0.0.1:43306)
0619 06:47:41.973961 (+ 4188us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:41.973982 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:41.974097 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:41.975230 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:41.975239 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:41.975273 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:47:41.975585 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:41.975594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.977352 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.977360 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:41.979012 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:41.979024 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.981123 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.981141 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:41.981342 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:41.981390 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 06:47:41.981529 (+ 139us) client_negotiation.cc:240] Negotiation successful
0619 06:47:41.981727 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4032,"thread_start_us":3866,"threads_started":1}
I20260619 06:47:41.983023 31021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:41.969492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:44827 (local address 127.0.0.1:54132)
0619 06:47:41.971724 (+ 2232us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:41.971761 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:41.971944 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:41.975806 (+ 3862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:41.975812 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:41.975841 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:47:41.976378 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:41.976392 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.978648 (+ 2256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.978653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:41.980566 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:41.980580 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.982470 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.982482 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:41.982608 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:41.982630 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 06:47:41.982681 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 06:47:41.982764 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2007,"mutex_wait_us":21,"thread_start_us":1870,"threads_started":1}
I20260619 06:47:41.983918 31022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:41.969730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54132 (local address 127.16.79.67:44827)
0619 06:47:41.975299 (+ 5569us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:41.975306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:41.975336 (+ 30us) server_negotiation.cc:407] Connection header received
0619 06:47:41.975419 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:41.975425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:41.975596 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:41.976590 (+ 994us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:47:41.976851 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.978501 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.980745 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.982008 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.983369 (+ 1361us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:41.983533 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:41.983583 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 06:47:41.983686 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5368,"thread_start_us":5191,"threads_started":1}
I20260619 06:47:41.986548 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:41.972837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:59692)
0619 06:47:41.974915 (+ 2078us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:41.974935 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:41.975136 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:41.975790 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:41.975794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:41.975822 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:47:41.976400 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:41.976414 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.978650 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.978653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:41.981666 (+ 3013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:41.981679 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.985736 (+ 4057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.985745 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:41.985931 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:41.985968 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 06:47:41.986051 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 06:47:41.986172 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1921,"mutex_wait_us":145,"thread_start_us":1774,"threads_started":1}
I20260619 06:47:41.986187 30811 tablet_service.cc:1467] Processing CreateTablet for tablet 1e2bb4137dd24cceaaf687342aa5b158 (DEFAULT_TABLE table=test-table [id=831533d95093457f8f8b1b9fff675591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:47:41.987316 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:41.972531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59692 (local address 127.16.79.65:42065)
0619 06:47:41.974413 (+ 1882us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:41.974424 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:41.975028 (+ 604us) server_negotiation.cc:407] Connection header received
0619 06:47:41.975376 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:41.975383 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:41.975603 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:41.975789 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:47:41.976873 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.978491 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.981830 (+ 3339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:41.985033 (+ 3203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:41.985180 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:41.985383 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:41.986819 (+ 1436us) server_negotiation.cc:299] Negotiation successful
0619 06:47:41.987001 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1666,"thread_start_us":1495,"threads_started":1}
I20260619 06:47:41.987790 30811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e2bb4137dd24cceaaf687342aa5b158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:47:41.987536 30944 tablet_service.cc:1467] Processing CreateTablet for tablet 1e2bb4137dd24cceaaf687342aa5b158 (DEFAULT_TABLE table=test-table [id=831533d95093457f8f8b1b9fff675591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:47:41.988988 30944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e2bb4137dd24cceaaf687342aa5b158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:47:41.990694 30677 tablet_service.cc:1467] Processing CreateTablet for tablet 1e2bb4137dd24cceaaf687342aa5b158 (DEFAULT_TABLE table=test-table [id=831533d95093457f8f8b1b9fff675591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:47:41.991958 30677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e2bb4137dd24cceaaf687342aa5b158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:47:42.000885 31027 tablet_bootstrap.cc:492] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b: Bootstrap starting.
I20260619 06:47:42.003376 31028 tablet_bootstrap.cc:492] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac: Bootstrap starting.
I20260619 06:47:42.004518 31027 tablet_bootstrap.cc:654] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b: Neither blocks nor log segments found. Creating new log.
I20260619 06:47:42.005905 31027 log.cc:824] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:42.005934 31029 tablet_bootstrap.cc:492] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2: Bootstrap starting.
I20260619 06:47:42.007092 31028 tablet_bootstrap.cc:654] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac: Neither blocks nor log segments found. Creating new log.
I20260619 06:47:42.008435 31028 log.cc:824] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:42.009511 31029 tablet_bootstrap.cc:654] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2: Neither blocks nor log segments found. Creating new log.
I20260619 06:47:42.009774 31027 tablet_bootstrap.cc:492] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b: No bootstrap required, opened a new log
I20260619 06:47:42.010057 31027 ts_tablet_manager.cc:1397] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260619 06:47:42.010735 31029 log.cc:824] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:42.011122 31028 tablet_bootstrap.cc:492] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac: No bootstrap required, opened a new log
I20260619 06:47:42.011389 31028 ts_tablet_manager.cc:1397] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.004s
I20260619 06:47:42.018802 31029 tablet_bootstrap.cc:492] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2: No bootstrap required, opened a new log
I20260619 06:47:42.019116 31029 ts_tablet_manager.cc:1397] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.004s
I20260619 06:47:42.019332 31028 raft_consensus.cc:348] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } } peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } }
I20260619 06:47:42.019687 31028 raft_consensus.cc:374] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:47:42.019793 31028 raft_consensus.cc:729] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e467369eecdd47158114ea2eb8bd20ac, State: Initialized, Role: FOLLOWER
I20260619 06:47:42.020421 31028 consensus_queue.cc:260] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } } peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } }
I20260619 06:47:42.021485 31009 heartbeater.cc:503] Master 127.16.79.126:46613 was elected leader, sending a full tablet report...
I20260619 06:47:42.021704 31027 raft_consensus.cc:348] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } } peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } }
I20260619 06:47:42.022104 31027 raft_consensus.cc:374] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:47:42.022104 31028 ts_tablet_manager.cc:1428] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260619 06:47:42.022187 31027 raft_consensus.cc:729] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d26783cc2ffe498d93143110c417d64b, State: Initialized, Role: FOLLOWER
I20260619 06:47:42.022749 31027 consensus_queue.cc:260] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } } peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } }
I20260619 06:47:42.023844 30876 heartbeater.cc:503] Master 127.16.79.126:46613 was elected leader, sending a full tablet report...
I20260619 06:47:42.024482 31027 ts_tablet_manager.cc:1428] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260619 06:47:42.029484 31029 raft_consensus.cc:348] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } } peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } }
I20260619 06:47:42.029814 31029 raft_consensus.cc:374] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:47:42.029886 31029 raft_consensus.cc:729] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be5258921314224a3abb9bec547f4f2, State: Initialized, Role: FOLLOWER
I20260619 06:47:42.030444 31029 consensus_queue.cc:260] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } } peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } }
I20260619 06:47:42.032217 31029 ts_tablet_manager.cc:1428] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260619 06:47:42.119048 31033 raft_consensus.cc:482] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:47:42.119347 31033 raft_consensus.cc:504] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } } peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } }
I20260619 06:47:42.121151 31033 leader_election.cc:288] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795), 7be5258921314224a3abb9bec547f4f2 (127.16.79.65:42065)
W20260619 06:47:42.125921 30743 tablet.cc:2369] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:47:42.127547 31011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:42.121321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:35795 (local address 127.16.79.67:41813)
0619 06:47:42.121596 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:42.121616 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:42.121756 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:42.122337 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:42.122345 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:42.122381 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:47:42.122585 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:42.122596 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:42.123869 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:42.123874 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:42.125393 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:42.125404 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:42.126785 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:42.126791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:42.126966 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:42.126989 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 06:47:42.127097 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 06:47:42.127243 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":1}
I20260619 06:47:42.127532 31024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:42.121718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:41813 (local address 127.16.79.66:35795)
0619 06:47:42.121926 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:42.121934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:42.121957 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:47:42.122046 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:42.122052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:42.122158 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:42.122265 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:47:42.122738 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:42.123718 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:42.125546 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:42.126632 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:42.126794 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:42.126869 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:42.127124 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 06:47:42.127257 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:47:42.128369 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:42.122249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:52039 (local address 127.16.79.65:42065)
0619 06:47:42.122473 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:42.122479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:42.122498 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:47:42.122616 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:42.122622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:42.122710 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:42.122801 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:47:42.123388 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:42.124318 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:42.126311 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:42.127366 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:42.127550 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:42.127621 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:42.127974 (+ 353us) server_negotiation.cc:299] Negotiation successful
0619 06:47:42.128075 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260619 06:47:42.128377 31036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:42.121566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.16.79.67:52039)
0619 06:47:42.122397 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:42.122415 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:42.122511 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:42.122846 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:42.122852 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:42.122875 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:47:42.123230 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:42.123244 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:42.124480 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:42.124484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:42.126133 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:42.126149 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:42.127570 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:42.127576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:42.127746 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:42.127775 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 06:47:42.127914 (+ 139us) client_negotiation.cc:240] Negotiation successful
0619 06:47:42.128045 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":435,"threads_started":1}
I20260619 06:47:42.128724 30831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e2bb4137dd24cceaaf687342aa5b158" candidate_uuid: "e467369eecdd47158114ea2eb8bd20ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d26783cc2ffe498d93143110c417d64b" is_pre_election: true
I20260619 06:47:42.129335 30831 raft_consensus.cc:2454] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e467369eecdd47158114ea2eb8bd20ac in term 0.
I20260619 06:47:42.129566 30697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e2bb4137dd24cceaaf687342aa5b158" candidate_uuid: "e467369eecdd47158114ea2eb8bd20ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be5258921314224a3abb9bec547f4f2" is_pre_election: true
I20260619 06:47:42.130031 30898 leader_election.cc:302] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d26783cc2ffe498d93143110c417d64b, e467369eecdd47158114ea2eb8bd20ac; no voters:
I20260619 06:47:42.130151 30697 raft_consensus.cc:2454] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e467369eecdd47158114ea2eb8bd20ac in term 0.
W20260619 06:47:42.130393 30877 tablet.cc:2369] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:47:42.130528 31033 raft_consensus.cc:2781] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:47:42.130693 31033 raft_consensus.cc:482] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:47:42.130822 31033 raft_consensus.cc:3037] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:47:42.133272 31033 raft_consensus.cc:504] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } } peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } }
I20260619 06:47:42.133893 31033 leader_election.cc:288] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [CANDIDATE]: Term 1 election: Requested vote from peers d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795), 7be5258921314224a3abb9bec547f4f2 (127.16.79.65:42065)
I20260619 06:47:42.134402 30697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e2bb4137dd24cceaaf687342aa5b158" candidate_uuid: "e467369eecdd47158114ea2eb8bd20ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7be5258921314224a3abb9bec547f4f2"
I20260619 06:47:42.134404 30831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e2bb4137dd24cceaaf687342aa5b158" candidate_uuid: "e467369eecdd47158114ea2eb8bd20ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d26783cc2ffe498d93143110c417d64b"
I20260619 06:47:42.134608 30697 raft_consensus.cc:3037] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:47:42.134624 30831 raft_consensus.cc:3037] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:47:42.136861 30697 raft_consensus.cc:2454] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e467369eecdd47158114ea2eb8bd20ac in term 1.
I20260619 06:47:42.136936 30831 raft_consensus.cc:2454] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e467369eecdd47158114ea2eb8bd20ac in term 1.
I20260619 06:47:42.137497 30898 leader_election.cc:302] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d26783cc2ffe498d93143110c417d64b, e467369eecdd47158114ea2eb8bd20ac; no voters:
I20260619 06:47:42.137838 31033 raft_consensus.cc:2781] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:47:42.138396 31033 raft_consensus.cc:686] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [term 1 LEADER]: Becoming Leader. State: Replica: e467369eecdd47158114ea2eb8bd20ac, State: Running, Role: LEADER
I20260619 06:47:42.138892 31033 consensus_queue.cc:237] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } } peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } }
I20260619 06:47:42.143812 30552 catalog_manager.cc:5495] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac reported cstate change: term changed from 0 to 1, leader changed from <none> to e467369eecdd47158114ea2eb8bd20ac (127.16.79.67). New cstate: current_term: 1 leader_uuid: "e467369eecdd47158114ea2eb8bd20ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } health_report { overall_health: UNKNOWN } } }
W20260619 06:47:42.146202 31010 tablet.cc:2369] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:47:42.171319 31022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:42.165903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54134 (local address 127.16.79.67:44827)
0619 06:47:42.166120 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:42.166125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:42.166145 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:47:42.166315 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:42.166319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:42.166399 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:42.166527 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:47:42.166889 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:42.167619 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:42.169096 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:42.169560 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:42.169705 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:42.170203 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:47:42.170750 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:47:42.170869 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:42.170934 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 06:47:42.171053 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 06:47:42.179849 30697 raft_consensus.cc:1261] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Refusing update from remote peer e467369eecdd47158114ea2eb8bd20ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:47:42.179849 30831 raft_consensus.cc:1261] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b [term 1 FOLLOWER]: Refusing update from remote peer e467369eecdd47158114ea2eb8bd20ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:47:42.180924 31037 consensus_queue.cc:1035] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:47:42.181303 31033 consensus_queue.cc:1035] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:47:42.207448 30552 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:47:42.207763 30552 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:47:42.211671 30552 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd: Sending DeleteTablet for 3 replicas of tablet 1e2bb4137dd24cceaaf687342aa5b158
I20260619 06:47:42.212963 30944 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e2bb4137dd24cceaaf687342aa5b158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:47:42 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:54132
I20260619 06:47:42.212963 30677 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e2bb4137dd24cceaaf687342aa5b158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:47:42 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:59692
I20260619 06:47:42.212965 30811 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e2bb4137dd24cceaaf687342aa5b158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:47:42 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:43306
I20260619 06:47:42.213447 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 30614
I20260619 06:47:42.223033 31052 tablet_replica.cc:330] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b: stopping tablet replica
I20260619 06:47:42.223590 31052 raft_consensus.cc:2229] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:47:42.224265 31052 raft_consensus.cc:2258] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:47:42.226625 31052 ts_tablet_manager.cc:1905] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:47:42.233428 31051 tablet_replica.cc:330] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac: stopping tablet replica
Jun 19 06:47:42 dist-test-slave-5w2x krb5kdc[30508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:47:42.234236 31051 raft_consensus.cc:2229] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:47:42.234728 31052 ts_tablet_manager.cc:1918] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:47:42.234951 31052 log.cc:1196] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/1e2bb4137dd24cceaaf687342aa5b158
I20260619 06:47:42.235311 31051 raft_consensus.cc:2258] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:47:42.235550 31052 ts_tablet_manager.cc:1939] T 1e2bb4137dd24cceaaf687342aa5b158 P d26783cc2ffe498d93143110c417d64b: Deleting consensus metadata
I20260619 06:47:42.237830 30538 catalog_manager.cc:4841] TS d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795): tablet 1e2bb4137dd24cceaaf687342aa5b158 (table test-table [id=831533d95093457f8f8b1b9fff675591]) successfully deleted
I20260619 06:47:42.238057 31050 tablet_replica.cc:330] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2: stopping tablet replica
I20260619 06:47:42.238544 31050 raft_consensus.cc:2229] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:47:42.239041 31050 raft_consensus.cc:2258] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:47:42.241111 31050 ts_tablet_manager.cc:1905] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:47:42.241120 31051 ts_tablet_manager.cc:1905] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:47:42.244462 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:42.214836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59704 (local address 127.16.79.65:42065)
0619 06:47:42.215011 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:42.215020 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:42.222753 (+ 7733us) server_negotiation.cc:407] Connection header received
0619 06:47:42.223179 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:42.223186 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:42.223273 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:42.223355 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:42.225680 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:42.226867 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:42.228830 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:42.229531 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:42.229631 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:42.235619 (+ 5988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:42.235644 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:42.235664 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:42.235723 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:42.240492 (+ 4769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:42.241575 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:42.241584 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:42.241589 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:42.241786 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:42.242302 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:42.242308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:42.242312 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:42.242789 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:42.243111 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:42.243747 (+ 636us) server_negotiation.cc:299] Negotiation successful
0619 06:47:42.244015 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 06:47:42.246307 30737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:42.249819 31051 ts_tablet_manager.cc:1918] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:47:42.249996 31051 log.cc:1196] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/1e2bb4137dd24cceaaf687342aa5b158
I20260619 06:47:42.250573 31051 ts_tablet_manager.cc:1939] T 1e2bb4137dd24cceaaf687342aa5b158 P e467369eecdd47158114ea2eb8bd20ac: Deleting consensus metadata
I20260619 06:47:42.252521 30539 catalog_manager.cc:4841] TS e467369eecdd47158114ea2eb8bd20ac (127.16.79.67:44827): tablet 1e2bb4137dd24cceaaf687342aa5b158 (table test-table [id=831533d95093457f8f8b1b9fff675591]) successfully deleted
I20260619 06:47:42.337261 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 30614
W20260619 06:47:42.360460 30539 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.79.65:42065 (error 108)
I20260619 06:47:42.360783 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 30748
I20260619 06:47:42.362510 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:42.362078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:59708)
0619 06:47:42.362244 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:42.362347 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 06:47:42.363933 30539 catalog_manager.cc:4584] TS 7be5258921314224a3abb9bec547f4f2 (127.16.79.65:42065): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e2bb4137dd24cceaaf687342aa5b158: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
Jun 19 06:47:42 dist-test-slave-5w2x krb5kdc[30508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:47:42.378587 31024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:42.361889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43322 (local address 127.16.79.66:35795)
0619 06:47:42.362068 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:42.362075 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:42.362096 (+ 21us) server_negotiation.cc:407] Connection header received
0619 06:47:42.362291 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:42.362298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:42.362399 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:42.362504 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:42.363865 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:42.365097 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:42.366380 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:42.367054 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:42.367183 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:42.371562 (+ 4379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:42.371580 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:42.371600 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:42.371648 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:42.375337 (+ 3689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:42.376004 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:42.376014 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:42.376019 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:42.376196 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:42.376660 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:42.376665 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:42.376668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:42.377171 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:42.377489 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:42.377799 (+ 310us) server_negotiation.cc:299] Negotiation successful
0619 06:47:42.378023 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:47:42.379352 30871 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:42.418066 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:42.417459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:59714)
0619 06:47:42.417729 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:42.417855 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260619 06:47:42.419303 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:42.418877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:59722)
0619 06:47:42.419049 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:42.419162 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 06:47:42.471880 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 30748
I20260619 06:47:42.480037 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:42.479438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:59734)
0619 06:47:42.479708 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:42.479822 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260619 06:47:42.481276 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:42.480872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:59748)
0619 06:47:42.481039 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:42.481127 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 06:47:42.486856 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 30881
Jun 19 06:47:42 dist-test-slave-5w2x krb5kdc[30508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:47:42.504685 31022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:42.487690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54140 (local address 127.16.79.67:44827)
0619 06:47:42.487878 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:42.487885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:42.487908 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:47:42.488091 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:42.488097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:42.488245 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:42.488357 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:42.489678 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:42.490910 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:42.492295 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:42.492982 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:42.493115 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:42.497948 (+ 4833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:42.497960 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:42.497978 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:42.498028 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:42.501463 (+ 3435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:42.502368 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:42.502378 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:42.502381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:42.502559 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:42.503001 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:42.503004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:42.503006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:42.503381 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:42.503606 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:42.504051 (+ 445us) server_negotiation.cc:299] Negotiation successful
0619 06:47:42.504291 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 06:47:42.505474 31004 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:42.562122 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:42.561536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:59756)
0619 06:47:42.561810 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:42.561929 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260619 06:47:42.563277 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:42.562820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:59762)
0619 06:47:42.563044 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:42.563127 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260619 06:47:42.627235 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 30881
I20260619 06:47:42.646734 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 30521
I20260619 06:47:42.647909 30583 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:42.762267 31026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:42.761700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:59778)
0619 06:47:42.761922 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:42.762024 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 06:47:42.762275 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 30521
I20260619 06:47:42.784446 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:46613
--webserver_interface=127.16.79.126
--webserver_port=35929
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:46613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:47:43.120728 31059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:43.121085 31059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:43.121258 31059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:43.128790 31059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:47:43.128938 31059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:43.129048 31059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:43.129153 31059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:47:43.129232 31059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:47:43.139417 31059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:46613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:46613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=35929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:43.140836 31059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:43.142161 31059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:43.165650 31068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:43.165767 31066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:43.165795 31065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:43.166706 31059 server_base.cc:1034] running on GCE node
I20260619 06:47:43.167694 31059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:43.169483 31059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:43.170733 31059 hybrid_clock.cc:648] HybridClock initialized: now 1781851663170678 us; error 45 us; skew 500 ppm
I20260619 06:47:43.171186 31059 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:43 dist-test-slave-5w2x krb5kdc[30508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851663, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:43.174885 31059 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:47:43.176636 31059 webserver.cc:460] Webserver started at http://127.16.79.126:35929/ using document root <none> and password file <none>
I20260619 06:47:43.177465 31059 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:43.177627 31059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:43.183485 31059 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 06:47:43.186515 31075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:43.188058 31059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:47:43.188309 31059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "6c3c2d38c4b74400a2979735183f3ffd"
format_stamp: "Formatted at 2026-06-19 06:47:40 on dist-test-slave-5w2x"
server_key: "e3231c3825d1c856da19060c846606ce"
server_key_iv: "f0e563c9e66bcdf90b230f0b45a3ea11"
server_key_version: "encryptionkey@0"
I20260619 06:47:43.189482 31059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:43.207269 31059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:43.211638 31059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:43.211947 31059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:43.232501 31059 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:46613
I20260619 06:47:43.232519 31126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:46613 every 8 connection(s)
I20260619 06:47:43.234190 31059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:47:43.237080 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 31059
I20260619 06:47:43.238107 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:42065
--local_ip_for_outbound_sockets=127.16.79.65
--tserver_master_addrs=127.16.79.126:46613
--webserver_port=39897
--webserver_interface=127.16.79.65
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:47:43.243790 31127 sys_catalog.cc:263] Verifying existing consensus state
I20260619 06:47:43.248430 31127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd: Bootstrap starting.
I20260619 06:47:43.271914 31127 log.cc:824] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:43.284341 31127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:47:43.284880 31127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd: Bootstrap complete.
I20260619 06:47:43.294550 31127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } }
I20260619 06:47:43.295804 31127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c3c2d38c4b74400a2979735183f3ffd, State: Initialized, Role: FOLLOWER
I20260619 06:47:43.296530 31127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } }
I20260619 06:47:43.296782 31127 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:47:43.296921 31127 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:47:43.297111 31127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 1 FOLLOWER]: Advancing to term 2
I20260619 06:47:43.301319 31127 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } }
I20260619 06:47:43.301877 31127 leader_election.cc:302] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c3c2d38c4b74400a2979735183f3ffd; no voters:
I20260619 06:47:43.302520 31127 leader_election.cc:288] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 06:47:43.302664 31131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 2 FOLLOWER]: Leader election won for term 2
I20260619 06:47:43.303974 31131 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 2 LEADER]: Becoming Leader. State: Replica: 6c3c2d38c4b74400a2979735183f3ffd, State: Running, Role: LEADER
I20260619 06:47:43.304618 31131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } }
I20260619 06:47:43.304992 31127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:47:43.307392 31133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c3c2d38c4b74400a2979735183f3ffd. Latest consensus state: current_term: 2 leader_uuid: "6c3c2d38c4b74400a2979735183f3ffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } } }
I20260619 06:47:43.307844 31133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [sys.catalog]: This master's current role is: LEADER
I20260619 06:47:43.308429 31132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6c3c2d38c4b74400a2979735183f3ffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } } }
I20260619 06:47:43.308753 31132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [sys.catalog]: This master's current role is: LEADER
I20260619 06:47:43.312741 31137 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:47:43.322707 31137 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:47:43.324793 31137 catalog_manager.cc:1259] Loaded cluster ID: ff4eef734e6246b3adabc2f4356d5de6
I20260619 06:47:43.325013 31137 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:47:43.329308 31137 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 06:47:43.355017 31137 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 06:47:43.372246 31137 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:47:43.374078 31137 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd: Loaded TSK: 0
I20260619 06:47:43.375658 31137 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 06:47:43.606238 31129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:43.606586 31129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:43.606760 31129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:43.614413 31129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:43.614537 31129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:43.614739 31129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:47:43.624993 31129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:42065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=39897
--tserver_master_addrs=127.16.79.126:46613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:43.626415 31129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:43.627780 31129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:43.642858 31154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:43.643357 31155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:43.644809 31157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:43.644909 31129 server_base.cc:1034] running on GCE node
I20260619 06:47:43.645664 31129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:43.646790 31129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:43.648053 31129 hybrid_clock.cc:648] HybridClock initialized: now 1781851663647976 us; error 65 us; skew 500 ppm
I20260619 06:47:43.648501 31129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:43 dist-test-slave-5w2x krb5kdc[30508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851663, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:43.651901 31129 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:47:43.653584 31129 webserver.cc:460] Webserver started at http://127.16.79.65:39897/ using document root <none> and password file <none>
I20260619 06:47:43.654330 31129 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:43.654481 31129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:43.660334 31129 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 06:47:43.663656 31164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:43.665241 31129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:47:43.665442 31129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "7be5258921314224a3abb9bec547f4f2"
format_stamp: "Formatted at 2026-06-19 06:47:40 on dist-test-slave-5w2x"
server_key: "126746a2345cf1b5b6408b53cd6f1562"
server_key_iv: "9bc70c4eb39fab32713840f6e4becc5c"
server_key_version: "encryptionkey@0"
I20260619 06:47:43.666452 31129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:43.687012 31129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:43.691628 31129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:43.691929 31129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:43.693118 31129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:47:43.695302 31171 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260619 06:47:43.700803 31171 ts_tablet_manager.cc:1852] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260619 06:47:43.701803 31171 ts_tablet_manager.cc:1905] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:47:43.709429 31171 ts_tablet_manager.cc:1918] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:47:43.709642 31171 log.cc:1196] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/1e2bb4137dd24cceaaf687342aa5b158
I20260619 06:47:43.710148 31171 ts_tablet_manager.cc:1939] T 1e2bb4137dd24cceaaf687342aa5b158 P 7be5258921314224a3abb9bec547f4f2: Deleting consensus metadata
I20260619 06:47:43.710590 31129 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260619 06:47:43.710713 31129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.016s user 0.001s sys 0.000s
I20260619 06:47:43.710860 31129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:47:43.710951 31129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:43.745216 31129 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:42065
I20260619 06:47:43.745301 31277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:42065 every 8 connection(s)
I20260619 06:47:43.746600 31129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:47:43.755424 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 31129
I20260619 06:47:43.756058 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:35795
--local_ip_for_outbound_sockets=127.16.79.66
--tserver_master_addrs=127.16.79.126:46613
--webserver_port=43011
--webserver_interface=127.16.79.66
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:47:43 dist-test-slave-5w2x krb5kdc[30508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851663, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:43.772326 31281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:43.749846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:52949)
0619 06:47:43.750662 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:43.750702 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:43.752518 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:43.753870 (+ 1352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:43.753877 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:43.754426 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:43.755132 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:43.755163 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:43.757053 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:43.757062 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:43.759237 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:43.759258 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:43.760336 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:43.760344 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:43.760667 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:43.761428 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:43.761469 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:43.764125 (+ 2656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:43.767598 (+ 3473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:43.767607 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:43.767622 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:43.768059 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:43.768591 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:43.768596 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:43.768600 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:43.768780 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:43.769548 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:43.769558 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:43.769927 (+ 369us) client_negotiation.cc:769] Sending connection context
0619 06:47:43.770288 (+ 361us) client_negotiation.cc:240] Negotiation successful
0619 06:47:43.770870 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":322,"threads_started":1}
I20260619 06:47:43.772313 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:43.749644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:52949 (local address 127.16.79.126:46613)
0619 06:47:43.750330 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:43.750346 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:43.750835 (+ 489us) server_negotiation.cc:407] Connection header received
0619 06:47:43.753045 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:43.753058 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:43.753391 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:43.753697 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:43.755417 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:43.756879 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:43.759393 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:43.760058 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:43.760270 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:43.764469 (+ 4199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:43.764477 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:43.764493 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:43.764531 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:43.767435 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:43.768266 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:43.768278 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:43.768284 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:43.768451 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:43.768911 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:43.768916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:43.768918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:43.769404 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:43.769641 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:43.770505 (+ 864us) server_negotiation.cc:299] Negotiation successful
0619 06:47:43.770890 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":263,"threads_started":1}
I20260619 06:47:43.775696 31278 heartbeater.cc:347] Connected to a master server at 127.16.79.126:46613
I20260619 06:47:43.776063 31278 heartbeater.cc:464] Registering TS with master...
I20260619 06:47:43.777020 31278 heartbeater.cc:511] Master 127.16.79.126:46613 requested a full tablet report, sending...
I20260619 06:47:43.779788 31092 ts_manager.cc:194] Registered new tserver with Master: 7be5258921314224a3abb9bec547f4f2 (127.16.79.65:42065)
I20260619 06:47:43.782043 31092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:52949
W20260619 06:47:44.092048 31282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:44.092394 31282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:44.092491 31282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:44.099620 31282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:44.099725 31282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:44.099843 31282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:47:44.109838 31282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:35795
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=43011
--tserver_master_addrs=127.16.79.126:46613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:44.111122 31282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:44.112440 31282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:44.125432 31287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:44.125335 31290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:44.125370 31288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:44.126307 31282 server_base.cc:1034] running on GCE node
I20260619 06:47:44.126904 31282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:44.127888 31282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:44.129129 31282 hybrid_clock.cc:648] HybridClock initialized: now 1781851664129055 us; error 82 us; skew 500 ppm
I20260619 06:47:44.129453 31282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:44 dist-test-slave-5w2x krb5kdc[30508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851664, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:44.132803 31282 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:47:44.134374 31282 webserver.cc:460] Webserver started at http://127.16.79.66:43011/ using document root <none> and password file <none>
I20260619 06:47:44.135078 31282 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:44.135244 31282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:44.140759 31282 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:47:44.143460 31297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:44.144836 31282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:47:44.145016 31282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "d26783cc2ffe498d93143110c417d64b"
format_stamp: "Formatted at 2026-06-19 06:47:41 on dist-test-slave-5w2x"
server_key: "26a07fd5fb6ac478f92645f17db1d4be"
server_key_iv: "6146becce51bf312741260063f6f575e"
server_key_version: "encryptionkey@0"
I20260619 06:47:44.146023 31282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:44.167529 31282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:44.171908 31282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:44.172263 31282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:44.173309 31282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:47:44.174679 31282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:47:44.174778 31282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:44.174938 31282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:47:44.175030 31282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:44.208642 31282 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:35795
I20260619 06:47:44.208738 31409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:35795 every 8 connection(s)
I20260619 06:47:44.210158 31282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:47:44.212577 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 31282
I20260619 06:47:44.213376 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:44827
--local_ip_for_outbound_sockets=127.16.79.67
--tserver_master_addrs=127.16.79.126:46613
--webserver_port=33735
--webserver_interface=127.16.79.67
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:47:44 dist-test-slave-5w2x krb5kdc[30508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851664, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:44.233767 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.212999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:47303 (local address 127.16.79.126:46613)
0619 06:47:44.213229 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:44.213236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:44.214816 (+ 1580us) server_negotiation.cc:407] Connection header received
0619 06:47:44.217724 (+ 2908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:44.217729 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:44.217803 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:44.217893 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:44.220103 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.220873 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.222457 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.223250 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.223396 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.227195 (+ 3799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:44.227200 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:44.227204 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:44.227238 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:44.229784 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:44.230732 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:44.230736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:44.230739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:44.230876 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:44.231392 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:44.231395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:44.231398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:44.231838 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:44.232011 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:44.233118 (+ 1107us) server_negotiation.cc:299] Negotiation successful
0619 06:47:44.233372 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 06:47:44.235255 31412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.213643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:47303)
0619 06:47:44.214612 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:44.214665 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:44.217395 (+ 2730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:44.218194 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:44.218211 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:44.219040 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:44.219893 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:44.219923 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.221049 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.221056 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:44.222302 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:44.222314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.223472 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.223477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:44.223669 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.224405 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:44.224432 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:44.226966 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:44.229925 (+ 2959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:44.229937 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:44.229958 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:44.230606 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:44.230994 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:44.231001 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:44.231008 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:44.231273 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:44.232086 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:44.232098 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:44.232635 (+ 537us) client_negotiation.cc:769] Sending connection context
0619 06:47:44.233002 (+ 367us) client_negotiation.cc:240] Negotiation successful
0619 06:47:44.233651 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":340,"threads_started":1}
I20260619 06:47:44.236944 31410 heartbeater.cc:347] Connected to a master server at 127.16.79.126:46613
I20260619 06:47:44.237320 31410 heartbeater.cc:464] Registering TS with master...
I20260619 06:47:44.238353 31410 heartbeater.cc:511] Master 127.16.79.126:46613 requested a full tablet report, sending...
I20260619 06:47:44.240272 31092 ts_manager.cc:194] Registered new tserver with Master: d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795)
I20260619 06:47:44.241475 31092 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:47303
W20260619 06:47:44.546108 31413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:44.546463 31413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:44.546641 31413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:44.554317 31413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:44.554497 31413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:44.554689 31413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:47:44.564762 31413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:44827
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=33735
--tserver_master_addrs=127.16.79.126:46613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:44.566188 31413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:44.567602 31413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:44.581429 31418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:44.581409 31419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:44.581490 31421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:44.582108 31413 server_base.cc:1034] running on GCE node
I20260619 06:47:44.582819 31413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:44.583833 31413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:44.585101 31413 hybrid_clock.cc:648] HybridClock initialized: now 1781851664585013 us; error 78 us; skew 500 ppm
I20260619 06:47:44.585549 31413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:44 dist-test-slave-5w2x krb5kdc[30508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851664, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:44.588953 31413 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:47:44.590431 31413 webserver.cc:460] Webserver started at http://127.16.79.67:33735/ using document root <none> and password file <none>
I20260619 06:47:44.591161 31413 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:44.591327 31413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:44.596843 31413 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:47:44.599561 31428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:44.600991 31413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 06:47:44.601174 31413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "e467369eecdd47158114ea2eb8bd20ac"
format_stamp: "Formatted at 2026-06-19 06:47:41 on dist-test-slave-5w2x"
server_key: "f723f5a77a09c70446ea167529533b3f"
server_key_iv: "567d31e6c0758d07d949229bdfd31881"
server_key_version: "encryptionkey@0"
I20260619 06:47:44.602211 31413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:44.621018 31413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:44.625612 31413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:44.625905 31413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:44.626899 31413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:47:44.628296 31413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:47:44.628422 31413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:44.628559 31413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:47:44.628647 31413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:44.662464 31413 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:44827
I20260619 06:47:44.662520 31540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:44827 every 8 connection(s)
I20260619 06:47:44.664007 31413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:47:44.669718 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 31413
Jun 19 06:47:44 dist-test-slave-5w2x krb5kdc[30508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851664, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:47:44.685091 31280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.666634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:34227 (local address 127.16.79.126:46613)
0619 06:47:44.666830 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:44.666835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:44.667989 (+ 1154us) server_negotiation.cc:407] Connection header received
0619 06:47:44.670000 (+ 2011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:44.670008 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:44.670095 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:44.670228 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:44.672000 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.673478 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.674985 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.675535 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.675724 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.679308 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:44.679313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:44.679316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:44.679348 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:44.682024 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:44.682730 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:44.682743 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:44.682746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:44.682873 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:44.683296 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:44.683302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:44.683315 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:44.683587 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:44.683745 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:44.684480 (+ 735us) server_negotiation.cc:299] Negotiation successful
0619 06:47:44.684699 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":78}
I20260619 06:47:44.686072 31543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.667143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:34227)
0619 06:47:44.667823 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:44.667867 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:44.669673 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:44.670477 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:44.670491 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:44.671277 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:44.671821 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:44.671837 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.673645 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.673649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:44.674845 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:44.674857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.675780 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.675788 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:44.676048 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.676803 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:44.676845 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:44.679098 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:44.682167 (+ 3069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:44.682176 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:44.682192 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:44.682604 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:44.682983 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:44.682991 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:44.682994 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:44.683174 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:44.683718 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:44.683726 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:44.684065 (+ 339us) client_negotiation.cc:769] Sending connection context
0619 06:47:44.684402 (+ 337us) client_negotiation.cc:240] Negotiation successful
0619 06:47:44.684914 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":268,"threads_started":1}
I20260619 06:47:44.687618 31541 heartbeater.cc:347] Connected to a master server at 127.16.79.126:46613
I20260619 06:47:44.687924 31541 heartbeater.cc:464] Registering TS with master...
I20260619 06:47:44.688665 31545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.670791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41922 (local address 127.16.79.126:46613)
0619 06:47:44.675277 (+ 4486us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:44.675281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:44.675295 (+ 14us) server_negotiation.cc:407] Connection header received
0619 06:47:44.675338 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:44.675341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:44.675398 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:44.675529 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:44.677408 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.678116 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.679560 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.680287 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.680432 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.683001 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:44.683006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:44.683009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:44.683038 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:44.685884 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:44.686637 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:44.686649 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:44.686651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:44.686775 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:44.687162 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:44.687167 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:44.687171 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:44.687450 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:44.687698 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:44.688027 (+ 329us) server_negotiation.cc:299] Negotiation successful
0619 06:47:44.688297 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":4391,"thread_start_us":4264,"threads_started":1}
I20260619 06:47:44.688863 31541 heartbeater.cc:511] Master 127.16.79.126:46613 requested a full tablet report, sending...
I20260619 06:47:44.690621 31091 ts_manager.cc:194] Registered new tserver with Master: e467369eecdd47158114ea2eb8bd20ac (127.16.79.67:44827)
I20260619 06:47:44.691638 31091 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:34227
I20260619 06:47:44.703768 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:47:44.710940 31545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.704714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41938 (local address 127.16.79.126:46613)
0619 06:47:44.704880 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:44.704885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:44.705459 (+ 574us) server_negotiation.cc:407] Connection header received
0619 06:47:44.705680 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:44.705683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:44.705828 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:44.705970 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:47:44.706425 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.707170 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.708749 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.709175 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.709326 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.709840 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:47:44.710371 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:47:44.710529 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:44.710585 (+ 56us) server_negotiation.cc:299] Negotiation successful
0619 06:47:44.710665 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:47:44.711822 31091 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:41938:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:47:44.719408 31091 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:47:44.750106 31554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.732945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38850 (local address 127.16.79.66:35795)
0619 06:47:44.734079 (+ 1134us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:44.734092 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:44.734747 (+ 655us) server_negotiation.cc:407] Connection header received
0619 06:47:44.735028 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:44.735037 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:44.735272 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:44.735459 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:47:44.736560 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.738330 (+ 1770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.741794 (+ 3464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.743502 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.743651 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.743945 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:44.749317 (+ 5372us) server_negotiation.cc:299] Negotiation successful
0619 06:47:44.749612 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":937,"thread_start_us":785,"threads_started":1}
I20260619 06:47:44.750851 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.731429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:32856)
0619 06:47:44.732989 (+ 1560us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:44.733040 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:44.733248 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:44.734147 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:44.734153 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:44.734186 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:47:44.734723 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:44.734739 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.737736 (+ 2997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.737745 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:44.741205 (+ 3460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:44.741229 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.744230 (+ 3001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.744241 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:44.747536 (+ 3295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.747617 (+ 81us) client_negotiation.cc:769] Sending connection context
0619 06:47:44.747767 (+ 150us) client_negotiation.cc:240] Negotiation successful
0619 06:47:44.748000 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1303,"mutex_wait_us":6,"thread_start_us":2931,"threads_started":1}
I20260619 06:47:44.751535 31551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.732231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:35795 (local address 127.0.0.1:38850)
0619 06:47:44.734632 (+ 2401us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:44.734655 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:44.734861 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:44.735803 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:44.735810 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:44.735847 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:47:44.736338 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:44.736354 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.738517 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.738522 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:44.741530 (+ 3008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:44.741552 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.744253 (+ 2701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.744260 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:44.748973 (+ 4713us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.749037 (+ 64us) client_negotiation.cc:769] Sending connection context
0619 06:47:44.750639 (+ 1602us) client_negotiation.cc:240] Negotiation successful
0619 06:47:44.750805 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2264,"mutex_wait_us":4372,"thread_start_us":644,"threads_started":1}
I20260619 06:47:44.751674 31552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.731624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32856 (local address 127.16.79.65:42065)
0619 06:47:44.732293 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:44.732299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:44.733504 (+ 1205us) server_negotiation.cc:407] Connection header received
0619 06:47:44.733629 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:44.733641 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:44.733940 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:44.734140 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:47:44.735147 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.737524 (+ 2377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.741466 (+ 3942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.743534 (+ 2068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.743697 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.743962 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:44.751071 (+ 7109us) server_negotiation.cc:299] Negotiation successful
0619 06:47:44.751269 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":1030,"threads_started":1}
I20260619 06:47:44.753826 31213 tablet_service.cc:1467] Processing CreateTablet for tablet fc71437e84a74254aff9fddbad600eeb (DEFAULT_TABLE table=test-table [id=4837017844c549b2a200109c1a0ed739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:47:44.755267 31213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc71437e84a74254aff9fddbad600eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:47:44.755577 31345 tablet_service.cc:1467] Processing CreateTablet for tablet fc71437e84a74254aff9fddbad600eeb (DEFAULT_TABLE table=test-table [id=4837017844c549b2a200109c1a0ed739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:47:44.757001 31345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc71437e84a74254aff9fddbad600eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:47:44.758734 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.735458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:44827 (local address 127.0.0.1:55966)
0619 06:47:44.751108 (+ 15650us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:44.751128 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:44.751247 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:44.752438 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:44.752442 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:44.752473 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:47:44.752658 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:44.752669 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.754606 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.754610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:44.756426 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:44.756440 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.757914 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.757921 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:44.758106 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.758144 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 06:47:44.758264 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 06:47:44.758412 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15549,"thread_start_us":15093,"threads_started":1}
I20260619 06:47:44.758788 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.735131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55966 (local address 127.16.79.67:44827)
0619 06:47:44.750378 (+ 15247us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:44.750387 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:44.751781 (+ 1394us) server_negotiation.cc:407] Connection header received
0619 06:47:44.751906 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:44.751915 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:44.752227 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:44.752430 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:47:44.752859 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.754432 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.756594 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.757738 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.757878 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.758107 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:44.758305 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 06:47:44.758458 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15056,"thread_start_us":15588,"threads_started":1}
I20260619 06:47:44.762230 31476 tablet_service.cc:1467] Processing CreateTablet for tablet fc71437e84a74254aff9fddbad600eeb (DEFAULT_TABLE table=test-table [id=4837017844c549b2a200109c1a0ed739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:47:44.763679 31476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc71437e84a74254aff9fddbad600eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:47:44.765581 31557 tablet_bootstrap.cc:492] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2: Bootstrap starting.
I20260619 06:47:44.769325 31557 tablet_bootstrap.cc:654] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2: Neither blocks nor log segments found. Creating new log.
I20260619 06:47:44.770639 31557 log.cc:824] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:44.773804 31559 tablet_bootstrap.cc:492] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b: Bootstrap starting.
I20260619 06:47:44.776054 31557 tablet_bootstrap.cc:492] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2: No bootstrap required, opened a new log
I20260619 06:47:44.776383 31557 ts_tablet_manager.cc:1397] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260619 06:47:44.777690 31559 tablet_bootstrap.cc:654] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b: Neither blocks nor log segments found. Creating new log.
I20260619 06:47:44.778954 31559 log.cc:824] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:44.781890 31559 tablet_bootstrap.cc:492] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b: No bootstrap required, opened a new log
I20260619 06:47:44.782158 31559 ts_tablet_manager.cc:1397] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260619 06:47:44.784768 31557 raft_consensus.cc:348] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } }
I20260619 06:47:44.785140 31557 raft_consensus.cc:374] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:47:44.785247 31557 raft_consensus.cc:729] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be5258921314224a3abb9bec547f4f2, State: Initialized, Role: FOLLOWER
I20260619 06:47:44.785836 31278 heartbeater.cc:503] Master 127.16.79.126:46613 was elected leader, sending a full tablet report...
I20260619 06:47:44.785894 31557 consensus_queue.cc:260] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } }
I20260619 06:47:44.787690 31557 ts_tablet_manager.cc:1428] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260619 06:47:44.788395 31561 tablet_bootstrap.cc:492] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac: Bootstrap starting.
I20260619 06:47:44.790612 31559 raft_consensus.cc:348] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } }
I20260619 06:47:44.791069 31559 raft_consensus.cc:374] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:47:44.791182 31559 raft_consensus.cc:729] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d26783cc2ffe498d93143110c417d64b, State: Initialized, Role: FOLLOWER
I20260619 06:47:44.791821 31559 consensus_queue.cc:260] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } }
I20260619 06:47:44.792233 31561 tablet_bootstrap.cc:654] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac: Neither blocks nor log segments found. Creating new log.
I20260619 06:47:44.792994 31410 heartbeater.cc:503] Master 127.16.79.126:46613 was elected leader, sending a full tablet report...
I20260619 06:47:44.793550 31561 log.cc:824] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:44.793629 31559 ts_tablet_manager.cc:1428] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260619 06:47:44.796615 31561 tablet_bootstrap.cc:492] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac: No bootstrap required, opened a new log
I20260619 06:47:44.796905 31561 ts_tablet_manager.cc:1397] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.006s
I20260619 06:47:44.803972 31561 raft_consensus.cc:348] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } }
I20260619 06:47:44.804440 31561 raft_consensus.cc:374] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:47:44.804562 31561 raft_consensus.cc:729] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e467369eecdd47158114ea2eb8bd20ac, State: Initialized, Role: FOLLOWER
I20260619 06:47:44.805130 31561 consensus_queue.cc:260] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } }
I20260619 06:47:44.806255 31541 heartbeater.cc:503] Master 127.16.79.126:46613 was elected leader, sending a full tablet report...
I20260619 06:47:44.806953 31561 ts_tablet_manager.cc:1428] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac: Time spent starting tablet: real 0.010s user 0.006s sys 0.004s
W20260619 06:47:44.915797 31542 tablet.cc:2369] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:47:44.935516 31562 raft_consensus.cc:482] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:47:44.935794 31562 raft_consensus.cc:504] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } }
I20260619 06:47:44.937462 31562 leader_election.cc:288] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e467369eecdd47158114ea2eb8bd20ac (127.16.79.67:44827), d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795)
I20260619 06:47:44.945937 31554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.937866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:39891 (local address 127.16.79.66:35795)
0619 06:47:44.938137 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:44.938146 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:44.938631 (+ 485us) server_negotiation.cc:407] Connection header received
0619 06:47:44.938840 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:44.938845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:44.938905 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:44.939003 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:47:44.939595 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.940670 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.943446 (+ 2776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.944695 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.944855 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.944928 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:44.945527 (+ 599us) server_negotiation.cc:299] Negotiation successful
0619 06:47:44.945667 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 06:47:44.945937 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.937915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:55133 (local address 127.16.79.67:44827)
0619 06:47:44.938211 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:44.938217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:44.938896 (+ 679us) server_negotiation.cc:407] Connection header received
0619 06:47:44.939096 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:44.939102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:44.939174 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:44.939254 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:47:44.939866 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.940925 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.943449 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.944707 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.944872 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.944935 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:44.945527 (+ 592us) server_negotiation.cc:299] Negotiation successful
0619 06:47:44.945667 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 06:47:44.946200 31567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.937812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:44827 (local address 127.16.79.65:55133)
0619 06:47:44.938781 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:44.938798 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:44.938912 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:44.939314 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:44.939318 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:44.939341 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:47:44.939705 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:44.939713 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.941072 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.941077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:44.943279 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:44.943293 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.944983 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.944991 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:44.945272 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.945394 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 06:47:44.945486 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 06:47:44.945682 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"mutex_wait_us":157,"thread_start_us":635,"threads_started":1}
I20260619 06:47:44.946236 31566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.937738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:35795 (local address 127.16.79.65:39891)
0619 06:47:44.938495 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:44.938518 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:44.938701 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:44.939018 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:44.939025 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:44.939050 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:47:44.939449 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:44.939462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.940833 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.940837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:44.943280 (+ 2443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:44.943293 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.944997 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.945002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:44.945270 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.945388 (+ 118us) client_negotiation.cc:769] Sending connection context
0619 06:47:44.945486 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 06:47:44.945682 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":167,"thread_start_us":414,"threads_started":1}
I20260619 06:47:44.947312 31365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc71437e84a74254aff9fddbad600eeb" candidate_uuid: "7be5258921314224a3abb9bec547f4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d26783cc2ffe498d93143110c417d64b" is_pre_election: true
I20260619 06:47:44.947324 31496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc71437e84a74254aff9fddbad600eeb" candidate_uuid: "7be5258921314224a3abb9bec547f4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e467369eecdd47158114ea2eb8bd20ac" is_pre_election: true
I20260619 06:47:44.947798 31496 raft_consensus.cc:2454] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7be5258921314224a3abb9bec547f4f2 in term 0.
I20260619 06:47:44.947808 31365 raft_consensus.cc:2454] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7be5258921314224a3abb9bec547f4f2 in term 0.
I20260619 06:47:44.948529 31167 leader_election.cc:302] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7be5258921314224a3abb9bec547f4f2, d26783cc2ffe498d93143110c417d64b, e467369eecdd47158114ea2eb8bd20ac; no voters:
I20260619 06:47:44.948951 31562 raft_consensus.cc:2781] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:47:44.949105 31562 raft_consensus.cc:482] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:47:44.949230 31562 raft_consensus.cc:3037] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:47:44.951448 31562 raft_consensus.cc:504] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } }
I20260619 06:47:44.952071 31562 leader_election.cc:288] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 1 election: Requested vote from peers e467369eecdd47158114ea2eb8bd20ac (127.16.79.67:44827), d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795)
I20260619 06:47:44.952533 31496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc71437e84a74254aff9fddbad600eeb" candidate_uuid: "7be5258921314224a3abb9bec547f4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e467369eecdd47158114ea2eb8bd20ac"
I20260619 06:47:44.952636 31365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc71437e84a74254aff9fddbad600eeb" candidate_uuid: "7be5258921314224a3abb9bec547f4f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d26783cc2ffe498d93143110c417d64b"
I20260619 06:47:44.952734 31496 raft_consensus.cc:3037] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:47:44.952893 31365 raft_consensus.cc:3037] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:47:44.955025 31496 raft_consensus.cc:2454] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be5258921314224a3abb9bec547f4f2 in term 1.
I20260619 06:47:44.955081 31365 raft_consensus.cc:2454] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7be5258921314224a3abb9bec547f4f2 in term 1.
I20260619 06:47:44.955472 31167 leader_election.cc:302] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7be5258921314224a3abb9bec547f4f2, e467369eecdd47158114ea2eb8bd20ac; no voters:
I20260619 06:47:44.955781 31562 raft_consensus.cc:2781] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:47:44.956284 31562 raft_consensus.cc:686] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 LEADER]: Becoming Leader. State: Replica: 7be5258921314224a3abb9bec547f4f2, State: Running, Role: LEADER
I20260619 06:47:44.956774 31562 consensus_queue.cc:237] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } }
I20260619 06:47:44.961464 31091 catalog_manager.cc:5495] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7be5258921314224a3abb9bec547f4f2 (127.16.79.65). New cstate: current_term: 1 leader_uuid: "7be5258921314224a3abb9bec547f4f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } health_report { overall_health: UNKNOWN } } }
W20260619 06:47:44.962144 31411 tablet.cc:2369] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:47:44.981866 31552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:44.976852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32870 (local address 127.16.79.65:42065)
0619 06:47:44.977026 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:44.977030 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:44.977048 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:47:44.977166 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:44.977170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:44.977252 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:44.977374 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:47:44.977794 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.978517 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.979924 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:44.980374 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:44.980466 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:44.980900 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:47:44.981397 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:47:44.981510 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:44.981574 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 06:47:44.981637 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:47:44.989889 31496 raft_consensus.cc:1261] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac [term 1 FOLLOWER]: Refusing update from remote peer 7be5258921314224a3abb9bec547f4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:47:44.989890 31365 raft_consensus.cc:1261] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b [term 1 FOLLOWER]: Refusing update from remote peer 7be5258921314224a3abb9bec547f4f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:47:44.990865 31562 consensus_queue.cc:1035] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:47:44.991143 31568 consensus_queue.cc:1035] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 06:47:44.999027 31279 tablet.cc:2369] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:47:45.002516 31571 mvcc.cc:205] Tried to move back new op lower bound from 7298464419788742656 to 7298464419666571264. Current Snapshot: MvccSnapshot[applied={T|T < 7298464419788742656}]
I20260619 06:47:45.019934 31090 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:41938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:47:45.020254 31090 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:41938:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:47:45.023801 31090 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd: Sending DeleteTablet for 3 replicas of tablet fc71437e84a74254aff9fddbad600eeb
I20260619 06:47:45.025099 31345 tablet_service.cc:1514] Processing DeleteTablet for tablet fc71437e84a74254aff9fddbad600eeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:47:45 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:38850
I20260619 06:47:45.025104 31213 tablet_service.cc:1514] Processing DeleteTablet for tablet fc71437e84a74254aff9fddbad600eeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:47:45 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:32856
I20260619 06:47:45.025305 31476 tablet_service.cc:1514] Processing DeleteTablet for tablet fc71437e84a74254aff9fddbad600eeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:47:45 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:55966
I20260619 06:47:45.026189 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 31129
I20260619 06:47:45.035967 31581 tablet_replica.cc:330] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b: stopping tablet replica
I20260619 06:47:45.036180 31583 tablet_replica.cc:330] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac: stopping tablet replica
I20260619 06:47:45.036862 31583 raft_consensus.cc:2229] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:47:45.037127 31581 raft_consensus.cc:2229] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:47:45.037531 31583 raft_consensus.cc:2258] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:47:45.038443 31582 tablet_replica.cc:330] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2: stopping tablet replica
I20260619 06:47:45.038897 31582 raft_consensus.cc:2229] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:47:45.039703 31582 raft_consensus.cc:2258] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:47:45.040409 31581 raft_consensus.cc:2258] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:47:45.040892 31583 ts_tablet_manager.cc:1905] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:47:45.042601 31581 ts_tablet_manager.cc:1905] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:47:45.043871 31582 ts_tablet_manager.cc:1905] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:47:45.047600 31552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:45.027743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32872 (local address 127.16.79.65:42065)
0619 06:47:45.027904 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:45.027909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:45.027926 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:47:45.027994 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:45.028000 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:45.028096 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:45.028227 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:45.030332 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:45.031592 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:45.033671 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:45.034353 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:45.034473 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:45.039061 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:45.039075 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:45.039092 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:45.039139 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:45.043232 (+ 4093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:45.044243 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:45.044251 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:45.044256 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:45.044434 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:45.044925 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:45.044931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:45.044934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:45.045381 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:45.045704 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:45.046980 (+ 1276us) server_negotiation.cc:299] Negotiation successful
0619 06:47:45.047222 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 06:47:45.047936 31583 ts_tablet_manager.cc:1918] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:47:45.048168 31583 log.cc:1196] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/fc71437e84a74254aff9fddbad600eeb
I20260619 06:47:45.048767 31273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:45.048775 31583 ts_tablet_manager.cc:1939] T fc71437e84a74254aff9fddbad600eeb P e467369eecdd47158114ea2eb8bd20ac: Deleting consensus metadata
I20260619 06:47:45.051270 31078 catalog_manager.cc:4841] TS e467369eecdd47158114ea2eb8bd20ac (127.16.79.67:44827): tablet fc71437e84a74254aff9fddbad600eeb (table test-table [id=4837017844c549b2a200109c1a0ed739]) successfully deleted
I20260619 06:47:45.052031 31581 ts_tablet_manager.cc:1918] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:47:45.052284 31581 log.cc:1196] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/fc71437e84a74254aff9fddbad600eeb
I20260619 06:47:45.052815 31581 ts_tablet_manager.cc:1939] T fc71437e84a74254aff9fddbad600eeb P d26783cc2ffe498d93143110c417d64b: Deleting consensus metadata
I20260619 06:47:45.054565 31077 catalog_manager.cc:4841] TS d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795): tablet fc71437e84a74254aff9fddbad600eeb (table test-table [id=4837017844c549b2a200109c1a0ed739]) successfully deleted
I20260619 06:47:45.148077 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 31129
I20260619 06:47:45.166606 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 31282
W20260619 06:47:45.166671 31078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.79.65:42065 (error 108)
I20260619 06:47:45.168598 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:45.168204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:32882)
0619 06:47:45.168345 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:45.168445 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 06:47:45.168861 31078 catalog_manager.cc:4584] TS 7be5258921314224a3abb9bec547f4f2 (127.16.79.65:42065): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fc71437e84a74254aff9fddbad600eeb: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
I20260619 06:47:45.181143 31554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:45.167560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38866 (local address 127.16.79.66:35795)
0619 06:47:45.167749 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:45.167757 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:45.167895 (+ 138us) server_negotiation.cc:407] Connection header received
0619 06:47:45.168096 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:45.168101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:45.168237 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:45.168341 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:45.169850 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:45.170583 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:45.171864 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:45.172371 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:45.172508 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:45.175135 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:45.175150 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:45.175165 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:45.175201 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:45.178029 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:45.179110 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:45.179135 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:45.179138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:45.179320 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:45.179682 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:45.179686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:45.179688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:45.180039 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:45.180333 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:45.180572 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 06:47:45.180741 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 06:47:45.181828 31405 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:45.222599 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:45.222057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:32890)
0619 06:47:45.222297 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:45.222420 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260619 06:47:45.223747 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:45.223305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:32894)
0619 06:47:45.223479 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:45.223575 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 06:47:45.277982 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 31282
I20260619 06:47:45.284335 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:45.283761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:32910)
0619 06:47:45.283997 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:45.284089 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260619 06:47:45.285400 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:45.285026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:32918)
0619 06:47:45.285178 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:45.285255 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 06:47:45.295992 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 31413
I20260619 06:47:45.310009 31555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:45.296872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55982 (local address 127.16.79.67:44827)
0619 06:47:45.297082 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:45.297087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:45.297106 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:47:45.297253 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:45.297257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:45.297328 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:45.297414 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:45.298751 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:45.299483 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:45.300785 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:45.301177 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:45.301285 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:45.303797 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:45.303813 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:45.303829 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:45.303865 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:45.307242 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:45.308034 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:45.308041 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:45.308043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:45.308267 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:45.308592 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:45.308595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:45.308598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:45.308949 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:45.309161 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:45.309463 (+ 302us) server_negotiation.cc:299] Negotiation successful
0619 06:47:45.309637 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 06:47:45.310644 31536 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:45.366158 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:45.365618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:32930)
0619 06:47:45.365893 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:45.365997 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260619 06:47:45.367362 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:45.366906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42065 (local address 127.0.0.1:32932)
0619 06:47:45.367124 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:45.367199 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260619 06:47:45.400859 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 31413
I20260619 06:47:45.417567 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 31059
I20260619 06:47:45.418761 31122 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:47:45.534317 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 31059
I20260619 06:47:45.556043 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:46613
--webserver_interface=127.16.79.126
--webserver_port=35929
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:46613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:47:45.882911 31589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:45.883196 31589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:45.883389 31589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:45.891599 31589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:47:45.891744 31589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:45.891819 31589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:45.891898 31589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:47:45.891965 31589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:47:45.902340 31589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:46613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:46613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=35929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:45.903628 31589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:45.905041 31589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:45.916781 31594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:45.916868 31597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:45.916805 31595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:45.917470 31589 server_base.cc:1034] running on GCE node
I20260619 06:47:45.918255 31589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:45.919595 31589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:45.920796 31589 hybrid_clock.cc:648] HybridClock initialized: now 1781851665920748 us; error 45 us; skew 500 ppm
I20260619 06:47:45.921152 31589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:45 dist-test-slave-5w2x krb5kdc[30508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851665, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:45.924268 31589 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:47:45.925709 31589 webserver.cc:460] Webserver started at http://127.16.79.126:35929/ using document root <none> and password file <none>
I20260619 06:47:45.926376 31589 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:45.926476 31589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:45.932293 31589 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 06:47:45.935163 31604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:45.936875 31589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:47:45.937073 31589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "6c3c2d38c4b74400a2979735183f3ffd"
format_stamp: "Formatted at 2026-06-19 06:47:40 on dist-test-slave-5w2x"
server_key: "e3231c3825d1c856da19060c846606ce"
server_key_iv: "f0e563c9e66bcdf90b230f0b45a3ea11"
server_key_version: "encryptionkey@0"
I20260619 06:47:45.938102 31589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:45.957764 31589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:45.962267 31589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:45.962566 31589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:45.982391 31589 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:46613
I20260619 06:47:45.982398 31655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:46613 every 8 connection(s)
I20260619 06:47:45.984022 31589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:47:45.988279 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 31589
I20260619 06:47:45.989115 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:42065
--local_ip_for_outbound_sockets=127.16.79.65
--tserver_master_addrs=127.16.79.126:46613
--webserver_port=39897
--webserver_interface=127.16.79.65
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:47:45.991382 31656 sys_catalog.cc:263] Verifying existing consensus state
I20260619 06:47:45.994978 31656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd: Bootstrap starting.
I20260619 06:47:46.026526 31656 log.cc:824] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:46.048843 31656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:47:46.049539 31656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd: Bootstrap complete.
I20260619 06:47:46.064613 31656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } }
I20260619 06:47:46.065907 31656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c3c2d38c4b74400a2979735183f3ffd, State: Initialized, Role: FOLLOWER
I20260619 06:47:46.066636 31656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } }
I20260619 06:47:46.066920 31656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:47:46.067042 31656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:47:46.067224 31656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 2 FOLLOWER]: Advancing to term 3
I20260619 06:47:46.072898 31656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } }
I20260619 06:47:46.073523 31656 leader_election.cc:302] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c3c2d38c4b74400a2979735183f3ffd; no voters:
I20260619 06:47:46.074229 31656 leader_election.cc:288] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 06:47:46.074410 31660 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 3 FOLLOWER]: Leader election won for term 3
I20260619 06:47:46.075630 31660 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [term 3 LEADER]: Becoming Leader. State: Replica: 6c3c2d38c4b74400a2979735183f3ffd, State: Running, Role: LEADER
I20260619 06:47:46.076262 31660 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } }
I20260619 06:47:46.076398 31656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:47:46.078720 31662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c3c2d38c4b74400a2979735183f3ffd. Latest consensus state: current_term: 3 leader_uuid: "6c3c2d38c4b74400a2979735183f3ffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } } }
I20260619 06:47:46.079077 31662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [sys.catalog]: This master's current role is: LEADER
I20260619 06:47:46.079705 31661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6c3c2d38c4b74400a2979735183f3ffd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c3c2d38c4b74400a2979735183f3ffd" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 46613 } } }
I20260619 06:47:46.080030 31661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c3c2d38c4b74400a2979735183f3ffd [sys.catalog]: This master's current role is: LEADER
I20260619 06:47:46.081274 31666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:47:46.089438 31666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:47:46.091082 31666 catalog_manager.cc:1259] Loaded cluster ID: ff4eef734e6246b3adabc2f4356d5de6
I20260619 06:47:46.091257 31666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 06:47:46.093338 31666 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f93bcb629ad google::LogMessage::Fail() at ??:0
@ 0x7f93bcb6588a google::LogMessage::SendToLog() at ??:0
@ 0x7f93bcb624de google::LogMessage::Flush() at ??:0
@ 0x7f93bcb63a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f93d2595512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f93d2592976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f93d261a610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f93d261a439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f93bf7ae345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f93bf7b191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f93bf7b1799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f93bf78ceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f93c2a666db start_thread at ??:0
@ 0x7f93bb9fc61f clone at ??:0
W20260619 06:47:46.417124 31658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:46.417425 31658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:46.417599 31658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:46.424924 31658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:46.425037 31658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:46.425318 31658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:47:46.435563 31658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:42065
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=39897
--tserver_master_addrs=127.16.79.126:46613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:46.436975 31658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:46.438374 31658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:46.451735 31695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:46.451810 31697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:46.451783 31694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:46.452632 31658 server_base.cc:1034] running on GCE node
I20260619 06:47:46.453286 31658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:46.454284 31658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:46.455544 31658 hybrid_clock.cc:648] HybridClock initialized: now 1781851666455485 us; error 46 us; skew 500 ppm
I20260619 06:47:46.455907 31658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:46 dist-test-slave-5w2x krb5kdc[30508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851666, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:46.459230 31658 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:47:46.460815 31658 webserver.cc:460] Webserver started at http://127.16.79.65:39897/ using document root <none> and password file <none>
I20260619 06:47:46.461514 31658 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:46.461666 31658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:46.467918 31658 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 06:47:46.472874 31704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:46.474354 31658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:47:46.474548 31658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "7be5258921314224a3abb9bec547f4f2"
format_stamp: "Formatted at 2026-06-19 06:47:40 on dist-test-slave-5w2x"
server_key: "126746a2345cf1b5b6408b53cd6f1562"
server_key_iv: "9bc70c4eb39fab32713840f6e4becc5c"
server_key_version: "encryptionkey@0"
I20260619 06:47:46.475744 31658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:46.494043 31658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:46.499114 31658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:46.499460 31658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:46.500576 31658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:47:46.502547 31711 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260619 06:47:46.508419 31658 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 06:47:46.508586 31658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260619 06:47:46.508749 31658 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 06:47:46.512712 31658 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 06:47:46.512832 31658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.004s sys 0.000s
I20260619 06:47:46.513195 31711 tablet_bootstrap.cc:492] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2: Bootstrap starting.
I20260619 06:47:46.521993 31711 log.cc:824] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2: Log is configured to *not* fsync() on all Append() calls
I20260619 06:47:46.538671 31711 tablet_bootstrap.cc:492] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:47:46.539601 31711 tablet_bootstrap.cc:492] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2: Bootstrap complete.
I20260619 06:47:46.540109 31711 ts_tablet_manager.cc:1397] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2: Time spent bootstrapping tablet: real 0.027s user 0.026s sys 0.000s
I20260619 06:47:46.551501 31711 raft_consensus.cc:348] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } }
I20260619 06:47:46.553376 31711 raft_consensus.cc:729] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be5258921314224a3abb9bec547f4f2, State: Initialized, Role: FOLLOWER
I20260619 06:47:46.554279 31711 consensus_queue.cc:260] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } }
I20260619 06:47:46.555222 31658 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:42065
I20260619 06:47:46.555279 31819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:42065 every 8 connection(s)
I20260619 06:47:46.556233 31711 ts_tablet_manager.cc:1428] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260619 06:47:46.556682 31658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:47:46.558718 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 31658
I20260619 06:47:46.559444 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:35795
--local_ip_for_outbound_sockets=127.16.79.66
--tserver_master_addrs=127.16.79.126:46613
--webserver_port=43011
--webserver_interface=127.16.79.66
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:47:46.562150 31822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:46.559852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:52485)
0619 06:47:46.560854 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:46.561152 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":439,"threads_started":1}
W20260619 06:47:46.563172 31820 heartbeater.cc:644] Failed to heartbeat to 127.16.79.126:46613 (0 consecutive failures): Network error: Failed to ping master at 127.16.79.126:46613: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
I20260619 06:47:46.564343 31822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:46.563870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:55613)
0619 06:47:46.564058 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:46.564218 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260619 06:47:46.565536 31822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:46.565013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:33513)
0619 06:47:46.565225 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:46.565339 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260619 06:47:46.565897 31820 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 06:47:46.895411 31823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:46.895854 31823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:46.896034 31823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:46.903702 31823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:46.903884 31823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:46.904065 31823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:47:46.914086 31823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:35795
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=43011
--tserver_master_addrs=127.16.79.126:46613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:46.915324 31823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:46.916697 31823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:46.929731 31828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:46.929762 31829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:46.929718 31831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:46.930502 31823 server_base.cc:1034] running on GCE node
I20260619 06:47:46.931143 31823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:46.932256 31823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:46.933506 31823 hybrid_clock.cc:648] HybridClock initialized: now 1781851666933454 us; error 40 us; skew 500 ppm
I20260619 06:47:46.933894 31823 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:46 dist-test-slave-5w2x krb5kdc[30508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851666, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:46.942085 31823 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:47:46.943810 31823 webserver.cc:460] Webserver started at http://127.16.79.66:43011/ using document root <none> and password file <none>
I20260619 06:47:46.944640 31823 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:46.944796 31823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:46.950711 31823 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 06:47:46.953933 31838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:46.955333 31823 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:47:46.955520 31823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "d26783cc2ffe498d93143110c417d64b"
format_stamp: "Formatted at 2026-06-19 06:47:41 on dist-test-slave-5w2x"
server_key: "26a07fd5fb6ac478f92645f17db1d4be"
server_key_iv: "6146becce51bf312741260063f6f575e"
server_key_version: "encryptionkey@0"
I20260619 06:47:46.956632 31823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:46.975358 31823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:46.979853 31823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:46.980227 31823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:46.981276 31823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:47:46.982616 31823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:47:46.982757 31823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:46.982892 31823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:47:46.982980 31823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:47.017364 31823 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:35795
I20260619 06:47:47.017454 31950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:35795 every 8 connection(s)
I20260619 06:47:47.018922 31823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:47:47.024062 31953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:47.022016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:42709)
0619 06:47:47.022766 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:47.022979 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":286,"threads_started":1}
I20260619 06:47:47.024111 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 31823
W20260619 06:47:47.024906 31951 heartbeater.cc:644] Failed to heartbeat to 127.16.79.126:46613 (0 consecutive failures): Network error: Failed to ping master at 127.16.79.126:46613: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
I20260619 06:47:47.024964 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:44827
--local_ip_for_outbound_sockets=127.16.79.67
--tserver_master_addrs=127.16.79.126:46613
--webserver_port=33735
--webserver_interface=127.16.79.67
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:46421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:47:47.025938 31953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:47.025459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:38585)
0619 06:47:47.025669 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:47.025774 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 06:47:47.026933 31953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:47.026502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:39899)
0619 06:47:47.026665 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:47.026774 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 06:47:47.027204 31951 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 06:47:47.351994 31954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:47:47.352368 31954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:47:47.352564 31954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:47:47.359788 31954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:47:47.359930 31954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:47:47.360138 31954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:47:47.370076 31954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:44827
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=33735
--tserver_master_addrs=127.16.79.126:46613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:47:47.371409 31954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:47:47.372771 31954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:47:47.385934 31959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:47.385841 31962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:47:47.385887 31960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:47:47.386824 31954 server_base.cc:1034] running on GCE node
I20260619 06:47:47.387449 31954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:47:47.388417 31954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:47:47.389635 31954 hybrid_clock.cc:648] HybridClock initialized: now 1781851667389591 us; error 37 us; skew 500 ppm
I20260619 06:47:47.389948 31954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:47:47 dist-test-slave-5w2x krb5kdc[30508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851667, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:47:47.393046 31954 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:47:47.394402 31954 webserver.cc:460] Webserver started at http://127.16.79.67:33735/ using document root <none> and password file <none>
I20260619 06:47:47.395082 31954 fs_manager.cc:359] Metadata directory not provided
I20260619 06:47:47.395226 31954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:47:47.400458 31954 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260619 06:47:47.403113 31969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:47.404527 31954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:47:47.404696 31954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "e467369eecdd47158114ea2eb8bd20ac"
format_stamp: "Formatted at 2026-06-19 06:47:41 on dist-test-slave-5w2x"
server_key: "f723f5a77a09c70446ea167529533b3f"
server_key_iv: "567d31e6c0758d07d949229bdfd31881"
server_key_version: "encryptionkey@0"
I20260619 06:47:47.405674 31954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:47:47.421499 31954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:47:47.425737 31954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:47:47.426023 31954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:47:47.426967 31954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:47:47.428319 31954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:47:47.428434 31954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:47.428566 31954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:47:47.428663 31954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:47:47.461685 31954 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:44827
I20260619 06:47:47.461743 32081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:44827 every 8 connection(s)
I20260619 06:47:47.463152 31954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:47:47.468089 32084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:47.466276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:41841)
0619 06:47:47.467022 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:47.467251 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":298,"threads_started":1}
I20260619 06:47:47.468448 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 31954
W20260619 06:47:47.468959 32082 heartbeater.cc:644] Failed to heartbeat to 127.16.79.126:46613 (0 consecutive failures): Network error: Failed to ping master at 127.16.79.126:46613: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
I20260619 06:47:47.469849 32084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:47.469509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:46545)
0619 06:47:47.469664 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:47.469745 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 06:47:47.470750 32084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:47.470408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:46001)
0619 06:47:47.470550 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:47.470619 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 06:47:47.470985 32082 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 06:47:47.567303 32086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:47.566470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:42785)
0619 06:47:47.566955 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:47.567089 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":240,"threads_started":1}
I20260619 06:47:48.028662 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:48.027815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:51675)
0619 06:47:48.028318 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:48.028443 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":210,"threads_started":1}
I20260619 06:47:48.325495 32088 raft_consensus.cc:482] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:47:48.325857 32088 raft_consensus.cc:504] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } }
I20260619 06:47:48.327454 32088 leader_election.cc:288] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e467369eecdd47158114ea2eb8bd20ac (127.16.79.67:44827), d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795)
Jun 19 06:47:48 dist-test-slave-5w2x krb5kdc[30508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851666, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
Jun 19 06:47:48 dist-test-slave-5w2x krb5kdc[30508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851666, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:47:48.364549 32089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:48.327751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:35795 (local address 127.16.79.65:43925)
0619 06:47:48.340279 (+ 12528us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:48.340341 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:48.341028 (+ 687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:48.343759 (+ 2731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:48.343769 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:48.344749 (+ 980us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:48.345434 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:48.345454 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:48.347851 (+ 2397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:48.347855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:48.349456 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:48.349469 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:48.350879 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:48.350885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:48.351202 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:48.352130 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:48.352190 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:48.355777 (+ 3587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:48.360055 (+ 4278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:48.360062 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:48.360075 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:48.360794 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:48.361265 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:48.361269 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:48.361272 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:48.361523 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:48.362620 (+ 1097us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:48.362628 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:48.363149 (+ 521us) client_negotiation.cc:769] Sending connection context
0619 06:47:48.363466 (+ 317us) client_negotiation.cc:240] Negotiation successful
0619 06:47:48.364040 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12399,"mutex_wait_us":179,"spinlock_wait_cycles":21376,"thread_start_us":12298,"threads_started":1}
I20260619 06:47:48.364641 32091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:48.328034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:54645 (local address 127.16.79.67:44827)
0619 06:47:48.340680 (+ 12646us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:48.340695 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:48.341316 (+ 621us) server_negotiation.cc:407] Connection header received
0619 06:47:48.342860 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:48.342868 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:48.343249 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:48.343607 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:48.345828 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:48.347622 (+ 1794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:48.349667 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:48.350625 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:48.350822 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:48.355672 (+ 4850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:48.355686 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:48.355708 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:48.355763 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:48.359865 (+ 4102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:48.360808 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:48.360818 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:48.360832 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:48.361076 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:48.361717 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:48.361723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:48.361726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:48.362459 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:48.362663 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:48.363757 (+ 1094us) server_negotiation.cc:299] Negotiation successful
0619 06:47:48.364223 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12475,"thread_start_us":12368,"threads_started":1}
I20260619 06:47:48.364641 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:48.328319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:43925 (local address 127.16.79.66:35795)
0619 06:47:48.339986 (+ 11667us) server_negotiation.cc:206] Beginning negotiation
0619 06:47:48.340010 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 06:47:48.340788 (+ 778us) server_negotiation.cc:407] Connection header received
0619 06:47:48.342837 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:47:48.342845 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:47:48.343245 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:47:48.343608 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:47:48.345710 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:48.347622 (+ 1912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:48.349603 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:47:48.350625 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:47:48.350818 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:48.356069 (+ 5251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:47:48.356076 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:47:48.356089 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:47:48.356130 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:47:48.359885 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:48.360951 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:48.360959 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:48.360965 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:48.361164 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:47:48.361737 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:47:48.361742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:47:48.361745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:47:48.362462 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:47:48.362663 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 06:47:48.363791 (+ 1128us) server_negotiation.cc:299] Negotiation successful
0619 06:47:48.364223 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11440,"thread_start_us":11264,"threads_started":1}
I20260619 06:47:48.364570 32090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:48.327767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:44827 (local address 127.16.79.65:54645)
0619 06:47:48.341173 (+ 13406us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:48.341189 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:47:48.341350 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:47:48.343755 (+ 2405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:47:48.343768 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:47:48.344770 (+ 1002us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:47:48.345631 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:48.345647 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:48.347846 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:48.347852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:48.349457 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:47:48.349470 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:47:48.350849 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:47:48.350860 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:47:48.351206 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:47:48.352130 (+ 924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:47:48.352198 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:47:48.355309 (+ 3111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:47:48.360094 (+ 4785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:48.360098 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:48.360101 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:48.360650 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:48.361210 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:47:48.361231 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:47:48.361249 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:47:48.361523 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:47:48.362620 (+ 1097us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:47:48.362628 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:47:48.363149 (+ 521us) client_negotiation.cc:769] Sending connection context
0619 06:47:48.363466 (+ 317us) client_negotiation.cc:240] Negotiation successful
0619 06:47:48.364190 (+ 724us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13270,"mutex_wait_us":559,"spinlock_wait_cycles":86528,"thread_start_us":13104,"threads_started":1}
W20260619 06:47:48.368647 31707 leader_election.cc:341] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e467369eecdd47158114ea2eb8bd20ac (127.16.79.67:44827): Not found: Tablet not found: fc71437e84a74254aff9fddbad600eeb
W20260619 06:47:48.369022 31706 leader_election.cc:341] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795): Not found: Tablet not found: fc71437e84a74254aff9fddbad600eeb
I20260619 06:47:48.369144 31706 leader_election.cc:302] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7be5258921314224a3abb9bec547f4f2; no voters: d26783cc2ffe498d93143110c417d64b, e467369eecdd47158114ea2eb8bd20ac
I20260619 06:47:48.369490 32088 raft_consensus.cc:2726] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:47:48.472437 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:48.471494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:53269)
0619 06:47:48.471992 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:48.472130 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":249,"threads_started":1}
I20260619 06:47:48.568904 32090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:48.568411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:41175)
0619 06:47:48.568597 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:48.568722 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260619 06:47:49.030148 32094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:49.029500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:44729)
0619 06:47:49.029924 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:49.029999 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":240,"threads_started":1}
I20260619 06:47:49.474051 32095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:49.473373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:53121)
0619 06:47:49.473802 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:49.473897 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":224,"threads_started":1}
I20260619 06:47:49.570817 32096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:49.569833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:45347)
0619 06:47:49.570369 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:49.570463 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":270,"threads_started":1}
I20260619 06:47:50.031821 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:50.031063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:33245)
0619 06:47:50.031553 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:50.031652 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":252,"threads_started":1}
I20260619 06:47:50.475690 32098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:50.474978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:50563)
0619 06:47:50.475418 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:50.475499 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":238,"threads_started":1}
I20260619 06:47:50.572515 32099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:50.571730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:60759)
0619 06:47:50.572257 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:50.572346 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":252,"threads_started":1}
I20260619 06:47:50.575524 32100 raft_consensus.cc:482] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:47:50.575757 32100 raft_consensus.cc:504] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } }
I20260619 06:47:50.576437 32100 leader_election.cc:288] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e467369eecdd47158114ea2eb8bd20ac (127.16.79.67:44827), d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795)
W20260619 06:47:50.577441 31707 leader_election.cc:341] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e467369eecdd47158114ea2eb8bd20ac (127.16.79.67:44827): Not found: Tablet not found: fc71437e84a74254aff9fddbad600eeb
W20260619 06:47:50.577646 31706 leader_election.cc:341] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795): Not found: Tablet not found: fc71437e84a74254aff9fddbad600eeb
I20260619 06:47:50.577785 31706 leader_election.cc:302] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7be5258921314224a3abb9bec547f4f2; no voters: d26783cc2ffe498d93143110c417d64b, e467369eecdd47158114ea2eb8bd20ac
I20260619 06:47:50.578033 32100 raft_consensus.cc:2726] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:47:51.033401 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:51.032684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:42117)
0619 06:47:51.033111 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:51.033191 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":216,"threads_started":1}
I20260619 06:47:51.477308 32102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:51.476654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:35671)
0619 06:47:51.477054 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:51.477132 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":203,"threads_started":1}
I20260619 06:47:51.574365 32103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:51.573595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:49327)
0619 06:47:51.574080 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:51.574167 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":215,"threads_started":1}
I20260619 06:47:52.035154 32104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:52.034412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:60721)
0619 06:47:52.034894 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:52.034983 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":267,"threads_started":1}
I20260619 06:47:52.478950 32105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:52.478231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:43133)
0619 06:47:52.478661 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:52.478760 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":239,"threads_started":1}
I20260619 06:47:52.576380 32106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:52.575472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:58781)
0619 06:47:52.576064 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:52.576210 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":341,"threads_started":1}
I20260619 06:47:53.037245 32107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:53.036318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:54007)
0619 06:47:53.036905 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:53.037015 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":336,"threads_started":1}
I20260619 06:47:53.312779 32108 raft_consensus.cc:482] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:47:53.312965 32108 raft_consensus.cc:504] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } }
I20260619 06:47:53.313561 32108 leader_election.cc:288] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e467369eecdd47158114ea2eb8bd20ac (127.16.79.67:44827), d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795)
W20260619 06:47:53.314658 31707 leader_election.cc:341] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e467369eecdd47158114ea2eb8bd20ac (127.16.79.67:44827): Not found: Tablet not found: fc71437e84a74254aff9fddbad600eeb
W20260619 06:47:53.314854 31706 leader_election.cc:341] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795): Not found: Tablet not found: fc71437e84a74254aff9fddbad600eeb
I20260619 06:47:53.314955 31706 leader_election.cc:302] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7be5258921314224a3abb9bec547f4f2; no voters: d26783cc2ffe498d93143110c417d64b, e467369eecdd47158114ea2eb8bd20ac
I20260619 06:47:53.315191 32108 raft_consensus.cc:2726] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:47:53.480729 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:53.479848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:54351)
0619 06:47:53.480407 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:53.480498 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":231,"threads_started":1}
I20260619 06:47:53.578146 32110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:53.577313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:47951)
0619 06:47:53.577772 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:53.577875 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":220,"threads_started":1}
I20260619 06:47:54.038901 32111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:54.038119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:45891)
0619 06:47:54.038601 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:54.038691 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":219,"threads_started":1}
I20260619 06:47:54.482445 32112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:54.481657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:36617)
0619 06:47:54.482179 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:54.482265 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":229,"threads_started":1}
I20260619 06:47:54.579782 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:54.579029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:52719)
0619 06:47:54.579470 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:54.579601 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":246,"threads_started":1}
I20260619 06:47:55.040591 32114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:55.039831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:49329)
0619 06:47:55.040307 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:55.040414 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":214,"threads_started":1}
I20260619 06:47:55.484325 32115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:55.483406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:43525)
0619 06:47:55.483947 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:55.484046 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":298,"threads_started":1}
I20260619 06:47:55.581562 32116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:55.580821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:39777)
0619 06:47:55.581290 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:55.581380 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":224,"threads_started":1}
I20260619 06:47:56.042366 32117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:56.041617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:60127)
0619 06:47:56.042087 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:56.042190 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":226,"threads_started":1}
I20260619 06:47:56.486109 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:56.485341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:39615)
0619 06:47:56.485849 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:56.485939 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":248,"threads_started":1}
I20260619 06:47:56.583387 32119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:56.582597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:58859)
0619 06:47:56.583137 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:56.583217 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":302,"threads_started":1}
I20260619 06:47:57.044579 32120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:57.043514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:59711)
0619 06:47:57.044182 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:57.044322 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":335,"threads_started":1}
I20260619 06:47:57.277407 32121 raft_consensus.cc:482] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:47:57.277676 32121 raft_consensus.cc:504] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } }
I20260619 06:47:57.278419 32121 leader_election.cc:288] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e467369eecdd47158114ea2eb8bd20ac (127.16.79.67:44827), d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795)
W20260619 06:47:57.279400 31706 leader_election.cc:341] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795): Not found: Tablet not found: fc71437e84a74254aff9fddbad600eeb
W20260619 06:47:57.279582 31707 leader_election.cc:341] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e467369eecdd47158114ea2eb8bd20ac (127.16.79.67:44827): Not found: Tablet not found: fc71437e84a74254aff9fddbad600eeb
I20260619 06:47:57.279726 31707 leader_election.cc:302] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7be5258921314224a3abb9bec547f4f2; no voters: d26783cc2ffe498d93143110c417d64b, e467369eecdd47158114ea2eb8bd20ac
I20260619 06:47:57.280004 32121 raft_consensus.cc:2726] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:47:57.487875 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:57.487081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:42093)
0619 06:47:57.487548 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:57.487634 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":253,"threads_started":1}
I20260619 06:47:57.585379 32123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:57.584396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:57041)
0619 06:47:57.585025 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:57.585125 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":363,"threads_started":1}
I20260619 06:47:58.046247 32124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:58.045453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:49315)
0619 06:47:58.045971 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:58.046056 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":279,"threads_started":1}
I20260619 06:47:58.489673 32125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:58.488910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:53847)
0619 06:47:58.489422 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:58.489514 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":221,"threads_started":1}
I20260619 06:47:58.587013 32126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:58.586309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:58441)
0619 06:47:58.586759 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:58.586846 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":240,"threads_started":1}
I20260619 06:47:59.047976 32127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:59.047267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:48923)
0619 06:47:59.047723 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:59.047808 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":233,"threads_started":1}
I20260619 06:47:59.491394 32128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:59.490681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:49853)
0619 06:47:59.491131 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:59.491226 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":241,"threads_started":1}
I20260619 06:47:59.588832 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:47:59.587997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:35221)
0619 06:47:59.588548 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0619 06:47:59.588649 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":292,"threads_started":1}
I20260619 06:48:00.049849 32130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:00.049091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:47551)
0619 06:48:00.049586 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:00.049684 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":240,"threads_started":1}
I20260619 06:48:00.493155 32131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:00.492445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:58359)
0619 06:48:00.492912 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:00.492998 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":245,"threads_started":1}
I20260619 06:48:00.590706 32132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:00.589871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:51559)
0619 06:48:00.590373 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:00.590471 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":274,"threads_started":1}
I20260619 06:48:01.051709 32133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:01.050893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:50537)
0619 06:48:01.051398 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:01.051488 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":244,"threads_started":1}
I20260619 06:48:01.491281 32134 raft_consensus.cc:482] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:48:01.491487 32134 raft_consensus.cc:504] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7be5258921314224a3abb9bec547f4f2" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42065 } } peers { permanent_uuid: "e467369eecdd47158114ea2eb8bd20ac" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 44827 } } peers { permanent_uuid: "d26783cc2ffe498d93143110c417d64b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 35795 } }
I20260619 06:48:01.492108 32134 leader_election.cc:288] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e467369eecdd47158114ea2eb8bd20ac (127.16.79.67:44827), d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795)
W20260619 06:48:01.493199 31706 leader_election.cc:341] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d26783cc2ffe498d93143110c417d64b (127.16.79.66:35795): Not found: Tablet not found: fc71437e84a74254aff9fddbad600eeb
W20260619 06:48:01.493374 31707 leader_election.cc:341] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e467369eecdd47158114ea2eb8bd20ac (127.16.79.67:44827): Not found: Tablet not found: fc71437e84a74254aff9fddbad600eeb
I20260619 06:48:01.493489 31707 leader_election.cc:302] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7be5258921314224a3abb9bec547f4f2; no voters: d26783cc2ffe498d93143110c417d64b, e467369eecdd47158114ea2eb8bd20ac
I20260619 06:48:01.493806 32134 raft_consensus.cc:2726] T fc71437e84a74254aff9fddbad600eeb P 7be5258921314224a3abb9bec547f4f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 06:48:01.494715 32135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:01.494053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:60205)
0619 06:48:01.494445 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:01.494532 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":197,"threads_started":1}
I20260619 06:48:01.592787 32136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:01.591924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.65:58711)
0619 06:48:01.592455 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:01.592594 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":284,"threads_started":1}
I20260619 06:48:02.053640 32137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:02.052850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.66:60943)
0619 06:48:02.053331 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:02.053465 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":248,"threads_started":1}
I20260619 06:48:02.474920 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 31658
I20260619 06:48:02.491065 32138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:02.476045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57360 (local address 127.16.79.65:42065)
0619 06:48:02.476839 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:02.476846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:02.476872 (+ 26us) server_negotiation.cc:407] Connection header received
0619 06:48:02.476941 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:02.476948 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:02.477250 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:02.477431 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:02.479730 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:02.480877 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:02.482143 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:02.482716 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:02.482865 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:02.485472 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:02.485497 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:02.485514 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:02.485548 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:02.488395 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:02.489236 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:02.489243 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:02.489245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:02.489381 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:02.489707 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:02.489710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:02.489712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:02.490051 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:02.490282 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:02.490582 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 06:48:02.490805 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":755,"threads_started":1}
I20260619 06:48:02.492376 31814 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:48:02.496793 32140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:02.495738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:46613 (local address 127.16.79.67:40691)
0619 06:48:02.496344 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:02.496529 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":245,"threads_started":1}
I20260619 06:48:02.584324 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 31658
I20260619 06:48:02.602231 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 31823
I20260619 06:48:02.616295 32141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:02.602995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36642 (local address 127.16.79.66:35795)
0619 06:48:02.603484 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:02.603488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:02.603506 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:48:02.603562 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:02.603565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:02.603625 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:02.603732 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:02.605243 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:02.606016 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:02.607241 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:02.607673 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:02.607767 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:02.610751 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:02.610756 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:02.610759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:02.610796 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:02.613461 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:02.614184 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:02.614189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:02.614193 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:02.614339 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:02.614730 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:02.614736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:02.614740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:02.615033 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:02.615315 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:02.615615 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 06:48:02.615843 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":292,"threads_started":1}
I20260619 06:48:02.617026 31946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:48:02.699857 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 31823
I20260619 06:48:02.717912 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 31954
I20260619 06:48:02.732060 32143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:02.718782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33960 (local address 127.16.79.67:44827)
0619 06:48:02.719341 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:02.719346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:02.719362 (+ 16us) server_negotiation.cc:407] Connection header received
0619 06:48:02.719415 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:02.719419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:02.719484 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:02.719621 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:02.721164 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:02.721955 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:02.723441 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:02.723872 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:02.724009 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:02.726687 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:02.726693 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:02.726696 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:02.726730 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:02.729295 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:02.729992 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:02.729999 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:02.730003 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:02.730156 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:02.730567 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:02.730574 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:02.730581 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:02.730852 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:02.731137 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:02.731443 (+ 306us) server_negotiation.cc:299] Negotiation successful
0619 06:48:02.731673 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":268,"threads_started":1}
I20260619 06:48:02.732828 32077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:48:02.817083 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 31954
2026-06-19T06:48:02Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24351 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:48:02 dist-test-slave-5w2x krb5kdc[32147](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:48:02 dist-test-slave-5w2x krb5kdc[32147](info): set up 2 sockets
Jun 19 06:48:02 dist-test-slave-5w2x krb5kdc[32147](info): commencing operation
krb5kdc: starting...
W20260619 06:48:03.936815 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.008s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:48:04 dist-test-slave-5w2x krb5kdc[32147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:48:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:48:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:48:04.179117 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:38789
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:38789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:48:04.510279 32160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:48:04.510560 32160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:48:04.510733 32160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:48:04.517954 32160 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 06:48:04.518059 32160 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 06:48:04.518129 32160 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 06:48:04.518213 32160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:48:04.518338 32160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:48:04.518448 32160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:48:04.518536 32160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:48:04.518625 32160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:48:04.528684 32160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:38789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:38789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:48:04.530010 32160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:48:04.531351 32160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:48:04.543558 32166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:04.543509 32168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:04.543463 32165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:48:04.544468 32160 server_base.cc:1034] running on GCE node
I20260619 06:48:04.545292 32160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:48:04.546937 32160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:48:04.548270 32160 hybrid_clock.cc:648] HybridClock initialized: now 1781851684548202 us; error 48 us; skew 500 ppm
I20260619 06:48:04.548626 32160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:48:04 dist-test-slave-5w2x krb5kdc[32147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851684, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:48:04.551884 32160 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:48:04.553553 32160 webserver.cc:460] Webserver started at http://127.16.79.126:46775/ using document root <none> and password file <none>
I20260619 06:48:04.554312 32160 fs_manager.cc:359] Metadata directory not provided
I20260619 06:48:04.554445 32160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:48:04.554778 32160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:48:04.557868 32160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "52da9733a5904a0d874bbdd4c4e3f23e"
format_stamp: "Formatted at 2026-06-19 06:48:04 on dist-test-slave-5w2x"
server_key: "ad5a977593c517ce564cf936b9fbac01"
server_key_iv: "60fddc4ebada38b4adf7bcdc30c16fc3"
server_key_version: "encryptionkey@0"
I20260619 06:48:04.558652 32160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "52da9733a5904a0d874bbdd4c4e3f23e"
format_stamp: "Formatted at 2026-06-19 06:48:04 on dist-test-slave-5w2x"
server_key: "ad5a977593c517ce564cf936b9fbac01"
server_key_iv: "60fddc4ebada38b4adf7bcdc30c16fc3"
server_key_version: "encryptionkey@0"
I20260619 06:48:04.563982 32160 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:48:04.567627 32175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:04.569075 32160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:48:04.569274 32160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "52da9733a5904a0d874bbdd4c4e3f23e"
format_stamp: "Formatted at 2026-06-19 06:48:04 on dist-test-slave-5w2x"
server_key: "ad5a977593c517ce564cf936b9fbac01"
server_key_iv: "60fddc4ebada38b4adf7bcdc30c16fc3"
server_key_version: "encryptionkey@0"
I20260619 06:48:04.569500 32160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:48:04.587165 32160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:48:04.591825 32160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:48:04.592121 32160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:48:04.611200 32160 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:38789
I20260619 06:48:04.611212 32226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:38789 every 8 connection(s)
I20260619 06:48:04.612672 32160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:48:04.616497 32227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:48:04.622376 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 32160
I20260619 06:48:04.622622 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:48:04.623525 16701 external_mini_cluster.cc:1428] Setting key 8770bd5fb9ef3de47c66d31c93d1862b
I20260619 06:48:04.628665 32227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Bootstrap starting.
I20260619 06:48:04.632846 32227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Neither blocks nor log segments found. Creating new log.
I20260619 06:48:04.634219 32227 log.cc:824] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Log is configured to *not* fsync() on all Append() calls
I20260619 06:48:04.637856 32227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: No bootstrap required, opened a new log
Jun 19 06:48:04 dist-test-slave-5w2x krb5kdc[32147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:48:04.648669 32227 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } }
I20260619 06:48:04.649134 32227 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:48:04.649242 32227 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52da9733a5904a0d874bbdd4c4e3f23e, State: Initialized, Role: FOLLOWER
I20260619 06:48:04.649925 32230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:04.625854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54938 (local address 127.16.79.126:38789)
0619 06:48:04.626686 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:04.626698 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:04.626738 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:48:04.628671 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:04.628696 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:04.629155 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:04.629507 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:04.631646 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:04.633161 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:04.635052 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:04.635995 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:04.636130 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:04.641114 (+ 4984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:04.641125 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:04.641150 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:04.641204 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:04.645250 (+ 4046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:04.645992 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:04.646001 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:04.646011 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:04.646249 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:04.646625 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:04.646630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:04.646633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:04.647392 (+ 759us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:04.647707 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:04.648031 (+ 324us) server_negotiation.cc:299] Negotiation successful
0619 06:48:04.648512 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":411,"threads_started":1}
I20260619 06:48:04.649948 32227 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } }
I20260619 06:48:04.650296 32227 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:48:04.650478 32227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:48:04.650679 32227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:48:04.653108 32227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } }
I20260619 06:48:04.653614 32227 leader_election.cc:302] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52da9733a5904a0d874bbdd4c4e3f23e; no voters:
I20260619 06:48:04.654183 32227 leader_election.cc:288] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:48:04.654320 32232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:48:04.654992 32232 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 1 LEADER]: Becoming Leader. State: Replica: 52da9733a5904a0d874bbdd4c4e3f23e, State: Running, Role: LEADER
I20260619 06:48:04.655557 32232 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } }
I20260619 06:48:04.656311 32227 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:48:04.658419 32234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52da9733a5904a0d874bbdd4c4e3f23e. Latest consensus state: current_term: 1 leader_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } } }
I20260619 06:48:04.658912 32234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [sys.catalog]: This master's current role is: LEADER
I20260619 06:48:04.660923 32240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:48:04.661012 32233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } } }
I20260619 06:48:04.661378 32233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [sys.catalog]: This master's current role is: LEADER
I20260619 06:48:04.666074 32240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:48:04.674983 32240 catalog_manager.cc:1347] Generated new cluster ID: 3a60bbf1772a4935ba299cdc98638fd3
I20260619 06:48:04.675091 32240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:48:04.707954 32240 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:48:04.709676 32240 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:48:04.718022 32240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Generated new TSK 0
I20260619 06:48:04.718874 32240 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:48:04.770787 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:38789
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:48:05.108668 32253 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:48:05.108953 32253 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:48:05.109107 32253 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:48:05.116492 32253 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:48:05.116611 32253 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:48:05.116794 32253 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:48:05.126731 32253 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:38789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:48:05.128156 32253 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:48:05.129580 32253 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:48:05.143563 32258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:05.143658 32261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:05.143613 32259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:48:05.144397 32253 server_base.cc:1034] running on GCE node
I20260619 06:48:05.145092 32253 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:48:05.146127 32253 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:48:05.147357 32253 hybrid_clock.cc:648] HybridClock initialized: now 1781851685147296 us; error 55 us; skew 500 ppm
I20260619 06:48:05.147722 32253 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:48:05 dist-test-slave-5w2x krb5kdc[32147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851685, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:48:05.151221 32253 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:48:05.152827 32253 webserver.cc:460] Webserver started at http://127.16.79.65:44917/ using document root <none> and password file <none>
I20260619 06:48:05.153519 32253 fs_manager.cc:359] Metadata directory not provided
I20260619 06:48:05.153625 32253 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:48:05.153884 32253 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:48:05.157133 32253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "cbdcc43755ca49d89776a664b86582b5"
format_stamp: "Formatted at 2026-06-19 06:48:05 on dist-test-slave-5w2x"
server_key: "132a17df790af093de9b09bb6cd82cef"
server_key_iv: "55588e7e7a7a900a37b136174fe28e93"
server_key_version: "encryptionkey@0"
I20260619 06:48:05.157902 32253 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "cbdcc43755ca49d89776a664b86582b5"
format_stamp: "Formatted at 2026-06-19 06:48:05 on dist-test-slave-5w2x"
server_key: "132a17df790af093de9b09bb6cd82cef"
server_key_iv: "55588e7e7a7a900a37b136174fe28e93"
server_key_version: "encryptionkey@0"
I20260619 06:48:05.163609 32253 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:48:05.167311 32268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:05.168820 32253 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 06:48:05.169015 32253 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "cbdcc43755ca49d89776a664b86582b5"
format_stamp: "Formatted at 2026-06-19 06:48:05 on dist-test-slave-5w2x"
server_key: "132a17df790af093de9b09bb6cd82cef"
server_key_iv: "55588e7e7a7a900a37b136174fe28e93"
server_key_version: "encryptionkey@0"
I20260619 06:48:05.169217 32253 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:48:05.183789 32253 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:48:05.188604 32253 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:48:05.188927 32253 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:48:05.189926 32253 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:48:05.191278 32253 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:48:05.191352 32253 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:05.191545 32253 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:48:05.191612 32253 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:05.225740 32253 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:42187
I20260619 06:48:05.225786 32380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:42187 every 8 connection(s)
I20260619 06:48:05.227067 32253 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:48:05.236083 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 32253
I20260619 06:48:05.236341 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:48:05.236989 16701 external_mini_cluster.cc:1428] Setting key 39003df55320dab9f4b1239146f206c5
Jun 19 06:48:05 dist-test-slave-5w2x krb5kdc[32147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851685, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:48:05.247789 32383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:05.229726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:57723 (local address 127.16.79.126:38789)
0619 06:48:05.230232 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:05.230237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:05.231059 (+ 822us) server_negotiation.cc:407] Connection header received
0619 06:48:05.233038 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:05.233043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:05.233144 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:05.233265 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:05.234821 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:05.235542 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:05.237329 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:05.237753 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:05.237889 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:05.241920 (+ 4031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:05.241925 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:05.241929 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:05.241962 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:05.244620 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:05.245380 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:05.245385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:05.245387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:05.245520 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:05.245978 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:05.245982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:05.245985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:05.246261 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:05.246421 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:05.247189 (+ 768us) server_negotiation.cc:299] Negotiation successful
0619 06:48:05.247424 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":273,"threads_started":1}
I20260619 06:48:05.248867 32384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:05.230152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:57723)
0619 06:48:05.230889 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:05.230928 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 06:48:05.232763 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:48:05.233520 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:48:05.233529 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:48:05.234074 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:48:05.234646 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:05.234664 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:05.235733 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:05.235737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:05.237181 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:05.237193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:05.238003 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:05.238011 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:05.238319 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:05.239125 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:48:05.239163 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:48:05.241683 (+ 2520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:48:05.244781 (+ 3098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:48:05.244790 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:48:05.244815 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:48:05.245240 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:48:05.245638 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:48:05.245643 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:48:05.245646 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:48:05.245847 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:48:05.246396 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:48:05.246403 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:48:05.246778 (+ 375us) client_negotiation.cc:769] Sending connection context
0619 06:48:05.247049 (+ 271us) client_negotiation.cc:240] Negotiation successful
0619 06:48:05.247632 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":288,"threads_started":1}
I20260619 06:48:05.250630 32381 heartbeater.cc:347] Connected to a master server at 127.16.79.126:38789
I20260619 06:48:05.250963 32381 heartbeater.cc:464] Registering TS with master...
I20260619 06:48:05.251717 32381 heartbeater.cc:511] Master 127.16.79.126:38789 requested a full tablet report, sending...
I20260619 06:48:05.254199 32192 ts_manager.cc:194] Registered new tserver with Master: cbdcc43755ca49d89776a664b86582b5 (127.16.79.65:42187)
I20260619 06:48:05.256495 32192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:57723
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:48:05.294955 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:38789
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:48:05.620831 32387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:48:05.621129 32387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:48:05.621289 32387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:48:05.628950 32387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:48:05.629081 32387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:48:05.629284 32387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:48:05.639564 32387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:38789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:48:05.640969 32387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:48:05.642334 32387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:48:05.655611 32392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:05.655642 32393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:05.655567 32395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:48:05.656844 32387 server_base.cc:1034] running on GCE node
I20260619 06:48:05.657577 32387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:48:05.658588 32387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:48:05.659825 32387 hybrid_clock.cc:648] HybridClock initialized: now 1781851685659772 us; error 45 us; skew 500 ppm
I20260619 06:48:05.660228 32387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:48:05 dist-test-slave-5w2x krb5kdc[32147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851685, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:48:05.663298 32387 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:48:05.664911 32387 webserver.cc:460] Webserver started at http://127.16.79.66:44745/ using document root <none> and password file <none>
I20260619 06:48:05.665630 32387 fs_manager.cc:359] Metadata directory not provided
I20260619 06:48:05.665781 32387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:48:05.666091 32387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:48:05.669169 32387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "4e6d13dc4160496bb87b022780c2ba8c"
format_stamp: "Formatted at 2026-06-19 06:48:05 on dist-test-slave-5w2x"
server_key: "1285c49c25acd9f6a5584d201254d13b"
server_key_iv: "d5b2ce0ffbad184be1e515b6db32458d"
server_key_version: "encryptionkey@0"
I20260619 06:48:05.669906 32387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "4e6d13dc4160496bb87b022780c2ba8c"
format_stamp: "Formatted at 2026-06-19 06:48:05 on dist-test-slave-5w2x"
server_key: "1285c49c25acd9f6a5584d201254d13b"
server_key_iv: "d5b2ce0ffbad184be1e515b6db32458d"
server_key_version: "encryptionkey@0"
I20260619 06:48:05.675448 32387 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:48:05.679416 32402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:05.680838 32387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:48:05.681015 32387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "4e6d13dc4160496bb87b022780c2ba8c"
format_stamp: "Formatted at 2026-06-19 06:48:05 on dist-test-slave-5w2x"
server_key: "1285c49c25acd9f6a5584d201254d13b"
server_key_iv: "d5b2ce0ffbad184be1e515b6db32458d"
server_key_version: "encryptionkey@0"
I20260619 06:48:05.681227 32387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:48:05.713735 32387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:48:05.717686 32387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:48:05.717991 32387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:48:05.718973 32387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:48:05.720356 32387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:48:05.720458 32387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:05.720618 32387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:48:05.720690 32387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:05.754356 32387 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:43495
I20260619 06:48:05.754401 32514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:43495 every 8 connection(s)
I20260619 06:48:05.755707 32387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:48:05.758755 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 32387
I20260619 06:48:05.758978 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:48:05.759447 16701 external_mini_cluster.cc:1428] Setting key 38afeeb60f86f3dc8f72670a387efb11
Jun 19 06:48:05 dist-test-slave-5w2x krb5kdc[32147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851685, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
I20260619 06:48:05.782261 32517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:05.758551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:55857 (local address 127.16.79.126:38789)
0619 06:48:05.759069 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:05.759075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:05.762158 (+ 3083us) server_negotiation.cc:407] Connection header received
0619 06:48:05.762216 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:05.762219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:05.762285 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:05.762406 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:05.764278 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:05.765020 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:05.766647 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:05.767072 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:05.767214 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:05.771040 (+ 3826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:05.771045 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:05.771049 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:05.771084 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:05.778859 (+ 7775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:05.779738 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:05.779751 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:05.779753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:05.779883 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:05.780439 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:05.780449 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:05.780452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:05.780754 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:05.780912 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:05.781767 (+ 855us) server_negotiation.cc:299] Negotiation successful
0619 06:48:05.781974 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":247,"threads_started":1}
I20260619 06:48:05.783247 32518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:05.759226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:55857)
0619 06:48:05.760051 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:05.760108 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 06:48:05.761865 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:48:05.762730 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:48:05.762740 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:48:05.763331 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:48:05.764041 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:05.764063 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:05.765207 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:05.765211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:05.766484 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:05.766496 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:05.767327 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:05.767334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:05.767566 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:05.768286 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:48:05.768308 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:48:05.770792 (+ 2484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:48:05.779060 (+ 8268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:48:05.779071 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:48:05.779103 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:48:05.779586 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:48:05.780006 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:48:05.780011 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:48:05.780014 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:48:05.780310 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:48:05.780888 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:48:05.780896 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:48:05.781280 (+ 384us) client_negotiation.cc:769] Sending connection context
0619 06:48:05.781572 (+ 292us) client_negotiation.cc:240] Negotiation successful
0619 06:48:05.782044 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":361,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
I20260619 06:48:05.784909 32515 heartbeater.cc:347] Connected to a master server at 127.16.79.126:38789
I20260619 06:48:05.785257 32515 heartbeater.cc:464] Registering TS with master...
I20260619 06:48:05.786217 32515 heartbeater.cc:511] Master 127.16.79.126:38789 requested a full tablet report, sending...
I20260619 06:48:05.787851 32192 ts_manager.cc:194] Registered new tserver with Master: 4e6d13dc4160496bb87b022780c2ba8c (127.16.79.66:43495)
I20260619 06:48:05.788805 32192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:55857
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:48:05.813277 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:38789
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:48:06.147289 32521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:48:06.147573 32521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:48:06.147732 32521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:48:06.155100 32521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:48:06.155213 32521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:48:06.155433 32521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:48:06.166010 32521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:38789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:48:06.167383 32521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:48:06.168880 32521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:48:06.184540 32526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:06.184491 32527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:06.185493 32529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:48:06.185437 32521 server_base.cc:1034] running on GCE node
I20260619 06:48:06.186256 32521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:48:06.187266 32521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:48:06.188478 32521 hybrid_clock.cc:648] HybridClock initialized: now 1781851686188436 us; error 37 us; skew 500 ppm
I20260619 06:48:06.188817 32521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:48:06 dist-test-slave-5w2x krb5kdc[32147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851686, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:48:06.192060 32521 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:48:06.193743 32521 webserver.cc:460] Webserver started at http://127.16.79.67:42645/ using document root <none> and password file <none>
I20260619 06:48:06.194427 32521 fs_manager.cc:359] Metadata directory not provided
I20260619 06:48:06.194563 32521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:48:06.194877 32521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:48:06.197894 32521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "d6945f02f2d54862aad0f78478f84b55"
format_stamp: "Formatted at 2026-06-19 06:48:06 on dist-test-slave-5w2x"
server_key: "3f63f8f37499d138b59893c8fe224e5a"
server_key_iv: "033b0713d9cd2921ef5c34bd2cd22efc"
server_key_version: "encryptionkey@0"
I20260619 06:48:06.198683 32521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "d6945f02f2d54862aad0f78478f84b55"
format_stamp: "Formatted at 2026-06-19 06:48:06 on dist-test-slave-5w2x"
server_key: "3f63f8f37499d138b59893c8fe224e5a"
server_key_iv: "033b0713d9cd2921ef5c34bd2cd22efc"
server_key_version: "encryptionkey@0"
I20260619 06:48:06.203795 32521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:48:06.207283 32536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:06.208676 32521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260619 06:48:06.208858 32521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "d6945f02f2d54862aad0f78478f84b55"
format_stamp: "Formatted at 2026-06-19 06:48:06 on dist-test-slave-5w2x"
server_key: "3f63f8f37499d138b59893c8fe224e5a"
server_key_iv: "033b0713d9cd2921ef5c34bd2cd22efc"
server_key_version: "encryptionkey@0"
I20260619 06:48:06.209060 32521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:48:06.232502 32521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:48:06.236634 32521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:48:06.236927 32521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:48:06.237905 32521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:48:06.239269 32521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:48:06.239346 32521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:06.239514 32521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:48:06.239565 32521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:06.260840 32381 heartbeater.cc:503] Master 127.16.79.126:38789 was elected leader, sending a full tablet report...
I20260619 06:48:06.274272 32521 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:40643
I20260619 06:48:06.274394 32648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:40643 every 8 connection(s)
I20260619 06:48:06.275636 32521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:48:06.276831 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 32521
I20260619 06:48:06.277055 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:48:06.277650 16701 external_mini_cluster.cc:1428] Setting key 1549d2d95eb3fb129fb2b9e2d4086470
Jun 19 06:48:06 dist-test-slave-5w2x krb5kdc[32147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851686, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:48:06.296432 32517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:06.278467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:49139 (local address 127.16.79.126:38789)
0619 06:48:06.278674 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:06.278680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:06.280051 (+ 1371us) server_negotiation.cc:407] Connection header received
0619 06:48:06.282252 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:06.282259 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:06.282366 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:06.282492 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:06.284133 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.285015 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.286516 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.286893 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.286982 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:06.290380 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:06.290384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:06.290388 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:06.290421 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:06.293776 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:06.294463 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:06.294466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:06.294468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:06.294584 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:06.294942 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:06.294945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:06.294947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:06.295219 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:06.295333 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:06.295975 (+ 642us) server_negotiation.cc:299] Negotiation successful
0619 06:48:06.296193 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 06:48:06.297549 32651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:06.279022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:49139)
0619 06:48:06.279864 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:06.279938 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0619 06:48:06.281908 (+ 1970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:48:06.282785 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:48:06.282799 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:48:06.283401 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:48:06.283978 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:06.283995 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.285151 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.285155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:06.286385 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:06.286396 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.287065 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.287070 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:06.287256 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:06.287936 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:48:06.287958 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:48:06.290180 (+ 2222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:48:06.293905 (+ 3725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:48:06.293913 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:48:06.293926 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:48:06.294365 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:48:06.294674 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:48:06.294677 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:48:06.294679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:48:06.294854 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:48:06.295322 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:48:06.295327 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:48:06.295655 (+ 328us) client_negotiation.cc:769] Sending connection context
0619 06:48:06.295885 (+ 230us) client_negotiation.cc:240] Negotiation successful
0619 06:48:06.296454 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":319,"threads_started":1}
I20260619 06:48:06.298893 32649 heartbeater.cc:347] Connected to a master server at 127.16.79.126:38789
I20260619 06:48:06.299213 32649 heartbeater.cc:464] Registering TS with master...
I20260619 06:48:06.299912 32649 heartbeater.cc:511] Master 127.16.79.126:38789 requested a full tablet report, sending...
I20260619 06:48:06.301394 32192 ts_manager.cc:194] Registered new tserver with Master: d6945f02f2d54862aad0f78478f84b55 (127.16.79.67:40643)
I20260619 06:48:06.302181 32192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:49139
I20260619 06:48:06.305744 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:48:06.322544 32517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:06.310056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54948 (local address 127.16.79.126:38789)
0619 06:48:06.310274 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:06.310283 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:06.310695 (+ 412us) server_negotiation.cc:407] Connection header received
0619 06:48:06.310856 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:06.310859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:06.310923 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:06.311003 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:06.312626 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.313500 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.314879 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.315271 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.315362 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:06.317670 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:06.317674 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:06.317678 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:06.317709 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:06.320204 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:06.320802 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:06.320805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:06.320808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:06.320933 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:06.321254 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:06.321257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:06.321259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:06.321562 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:06.321761 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:06.322029 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 06:48:06.322202 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:48:06.326719 32192 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:48:06.333746 32192 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:48:06.358573 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:06.347600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40010 (local address 127.16.79.65:42187)
0619 06:48:06.350889 (+ 3289us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:06.350899 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:06.350928 (+ 29us) server_negotiation.cc:407] Connection header received
0619 06:48:06.351035 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:06.351042 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:06.351275 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:06.351507 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:48:06.352216 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.353793 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.355847 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.357403 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.357555 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:06.357866 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:06.358082 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 06:48:06.358245 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3118,"thread_start_us":2927,"threads_started":1}
I20260619 06:48:06.358726 32661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:06.347310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42187 (local address 127.0.0.1:40010)
0619 06:48:06.348080 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:06.348117 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:48:06.348316 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:48:06.351503 (+ 3187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:48:06.351508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:48:06.351534 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:48:06.351998 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:06.352011 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.353928 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.353938 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:06.355620 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:06.355632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.357600 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.357606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:06.357803 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:06.357861 (+ 58us) client_negotiation.cc:769] Sending connection context
0619 06:48:06.358028 (+ 167us) client_negotiation.cc:240] Negotiation successful
0619 06:48:06.358224 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":7,"thread_start_us":366,"threads_started":1}
I20260619 06:48:06.359445 32663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:06.348601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:43495 (local address 127.0.0.1:54702)
0619 06:48:06.350230 (+ 1629us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:06.350246 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:48:06.350342 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:48:06.351151 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:48:06.351155 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:48:06.351180 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:48:06.351627 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:06.351644 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.353638 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.353643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:06.356276 (+ 2633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:06.356293 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.358465 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.358483 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:06.358742 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:06.358778 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 06:48:06.358869 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 06:48:06.359005 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1531,"mutex_wait_us":133,"thread_start_us":300,"threads_started":1}
I20260619 06:48:06.359472 32664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:06.347889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:40643 (local address 127.0.0.1:43958)
0619 06:48:06.349161 (+ 1272us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:06.349179 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:48:06.349292 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:48:06.350806 (+ 1514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:48:06.350813 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:48:06.350844 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:48:06.351280 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:06.351310 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.353478 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.353484 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:06.356387 (+ 2903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:06.356399 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.358480 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.358486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:06.358873 (+ 387us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:06.358903 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 06:48:06.358971 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 06:48:06.359091 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1148,"mutex_wait_us":173,"thread_start_us":2089,"threads_started":1}
I20260619 06:48:06.360070 32666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:06.348843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54702 (local address 127.16.79.66:43495)
0619 06:48:06.350612 (+ 1769us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:06.350621 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:06.350649 (+ 28us) server_negotiation.cc:407] Connection header received
0619 06:48:06.350760 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:06.350774 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:06.350959 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:06.351171 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:48:06.351859 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.353489 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.356490 (+ 3001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.357950 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.358087 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:06.358287 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:06.359227 (+ 940us) server_negotiation.cc:299] Negotiation successful
0619 06:48:06.359387 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1627,"thread_start_us":1504,"threads_started":1}
I20260619 06:48:06.360147 32665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:06.348667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43958 (local address 127.16.79.67:40643)
0619 06:48:06.350067 (+ 1400us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:06.350078 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:06.350113 (+ 35us) server_negotiation.cc:407] Connection header received
0619 06:48:06.350217 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:06.350226 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:06.350539 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:06.351501 (+ 962us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:48:06.351795 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.353348 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.356563 (+ 3215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.357950 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.358087 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:06.358310 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:06.359566 (+ 1256us) server_negotiation.cc:299] Negotiation successful
0619 06:48:06.359759 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1113,"thread_start_us":949,"threads_started":1}
I20260619 06:48:06.363353 32450 tablet_service.cc:1467] Processing CreateTablet for tablet 6e81d2483c224b2980f127a4dec4ad24 (DEFAULT_TABLE table=test-table [id=9a294cdd69254b2aba31fb17d682e71b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:48:06.363248 32316 tablet_service.cc:1467] Processing CreateTablet for tablet 6e81d2483c224b2980f127a4dec4ad24 (DEFAULT_TABLE table=test-table [id=9a294cdd69254b2aba31fb17d682e71b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:48:06.363613 32584 tablet_service.cc:1467] Processing CreateTablet for tablet 6e81d2483c224b2980f127a4dec4ad24 (DEFAULT_TABLE table=test-table [id=9a294cdd69254b2aba31fb17d682e71b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:48:06.364734 32450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e81d2483c224b2980f127a4dec4ad24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:48:06.364825 32584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e81d2483c224b2980f127a4dec4ad24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:48:06.364825 32316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e81d2483c224b2980f127a4dec4ad24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:48:06.382717 32668 tablet_bootstrap.cc:492] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5: Bootstrap starting.
I20260619 06:48:06.386310 32667 tablet_bootstrap.cc:492] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c: Bootstrap starting.
I20260619 06:48:06.386967 32669 tablet_bootstrap.cc:492] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55: Bootstrap starting.
I20260619 06:48:06.387277 32668 tablet_bootstrap.cc:654] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5: Neither blocks nor log segments found. Creating new log.
I20260619 06:48:06.388700 32668 log.cc:824] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5: Log is configured to *not* fsync() on all Append() calls
I20260619 06:48:06.389724 32667 tablet_bootstrap.cc:654] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c: Neither blocks nor log segments found. Creating new log.
I20260619 06:48:06.390581 32669 tablet_bootstrap.cc:654] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55: Neither blocks nor log segments found. Creating new log.
I20260619 06:48:06.390738 32667 log.cc:824] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c: Log is configured to *not* fsync() on all Append() calls
I20260619 06:48:06.392020 32669 log.cc:824] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55: Log is configured to *not* fsync() on all Append() calls
I20260619 06:48:06.395144 32669 tablet_bootstrap.cc:492] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55: No bootstrap required, opened a new log
I20260619 06:48:06.395144 32668 tablet_bootstrap.cc:492] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5: No bootstrap required, opened a new log
I20260619 06:48:06.395318 32667 tablet_bootstrap.cc:492] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c: No bootstrap required, opened a new log
I20260619 06:48:06.395424 32668 ts_tablet_manager.cc:1397] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260619 06:48:06.395424 32669 ts_tablet_manager.cc:1397] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260619 06:48:06.395614 32667 ts_tablet_manager.cc:1397] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260619 06:48:06.403017 32667 raft_consensus.cc:348] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } }
I20260619 06:48:06.403409 32667 raft_consensus.cc:374] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:48:06.403506 32667 raft_consensus.cc:729] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e6d13dc4160496bb87b022780c2ba8c, State: Initialized, Role: FOLLOWER
I20260619 06:48:06.404176 32667 consensus_queue.cc:260] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } }
I20260619 06:48:06.406458 32668 raft_consensus.cc:348] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } }
I20260619 06:48:06.406458 32669 raft_consensus.cc:348] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } }
I20260619 06:48:06.406908 32669 raft_consensus.cc:374] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:48:06.406908 32668 raft_consensus.cc:374] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:48:06.407016 32668 raft_consensus.cc:729] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbdcc43755ca49d89776a664b86582b5, State: Initialized, Role: FOLLOWER
I20260619 06:48:06.407016 32669 raft_consensus.cc:729] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6945f02f2d54862aad0f78478f84b55, State: Initialized, Role: FOLLOWER
I20260619 06:48:06.407789 32515 heartbeater.cc:503] Master 127.16.79.126:38789 was elected leader, sending a full tablet report...
I20260619 06:48:06.407758 32669 consensus_queue.cc:260] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } }
I20260619 06:48:06.407758 32668 consensus_queue.cc:260] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } }
I20260619 06:48:06.408702 32667 ts_tablet_manager.cc:1428] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260619 06:48:06.410557 32669 ts_tablet_manager.cc:1428] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55: Time spent starting tablet: real 0.015s user 0.013s sys 0.003s
I20260619 06:48:06.410837 32668 ts_tablet_manager.cc:1428] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5: Time spent starting tablet: real 0.015s user 0.013s sys 0.003s
I20260619 06:48:06.411540 32649 heartbeater.cc:503] Master 127.16.79.126:38789 was elected leader, sending a full tablet report...
W20260619 06:48:06.479441 32382 tablet.cc:2369] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:48:06.508021 32516 tablet.cc:2369] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:48:06.527966 32650 tablet.cc:2369] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:48:06.705745 32674 raft_consensus.cc:482] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:48:06.706112 32674 raft_consensus.cc:504] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } }
I20260619 06:48:06.707973 32674 leader_election.cc:288] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbdcc43755ca49d89776a664b86582b5 (127.16.79.65:42187), 4e6d13dc4160496bb87b022780c2ba8c (127.16.79.66:43495)
I20260619 06:48:06.714650 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:06.708517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:34705 (local address 127.16.79.65:42187)
0619 06:48:06.708779 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:06.708785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:06.708808 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:48:06.708910 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:06.708916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:06.709014 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:06.709132 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:48:06.709569 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.710563 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.712769 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.713671 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.713878 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:06.713932 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:06.714272 (+ 340us) server_negotiation.cc:299] Negotiation successful
0619 06:48:06.714400 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260619 06:48:06.714687 32651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:06.708428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42187 (local address 127.16.79.67:34705)
0619 06:48:06.708643 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:06.708659 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:48:06.708761 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:48:06.709172 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:48:06.709176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:48:06.709207 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:48:06.709418 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:06.709428 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.710692 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.710696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:06.712576 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:06.712592 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.713880 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.713886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:06.714090 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:06.714129 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 06:48:06.714228 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 06:48:06.714366 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":1}
I20260619 06:48:06.715838 32666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:06.709121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:40413 (local address 127.16.79.66:43495)
0619 06:48:06.709315 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:06.709320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:06.709779 (+ 459us) server_negotiation.cc:407] Connection header received
0619 06:48:06.709887 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:06.709893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:06.709966 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:06.710056 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:48:06.710494 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.711304 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.713406 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.714511 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.714714 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:06.714789 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:06.715435 (+ 646us) server_negotiation.cc:299] Negotiation successful
0619 06:48:06.715577 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:48:06.715883 32676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:06.708865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:43495 (local address 127.16.79.67:40413)
0619 06:48:06.709679 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:06.709693 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 06:48:06.709790 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:48:06.710077 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:48:06.710080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:48:06.710098 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:48:06.710358 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:06.710366 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.711470 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.711475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:06.713257 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:06.713268 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.715098 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.715104 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:06.715288 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:06.715320 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 06:48:06.715408 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 06:48:06.715555 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":574,"threads_started":1}
I20260619 06:48:06.715737 32336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e81d2483c224b2980f127a4dec4ad24" candidate_uuid: "d6945f02f2d54862aad0f78478f84b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbdcc43755ca49d89776a664b86582b5" is_pre_election: true
I20260619 06:48:06.716338 32336 raft_consensus.cc:2454] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6945f02f2d54862aad0f78478f84b55 in term 0.
I20260619 06:48:06.717087 32540 leader_election.cc:302] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbdcc43755ca49d89776a664b86582b5, d6945f02f2d54862aad0f78478f84b55; no voters:
I20260619 06:48:06.717015 32470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e81d2483c224b2980f127a4dec4ad24" candidate_uuid: "d6945f02f2d54862aad0f78478f84b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e6d13dc4160496bb87b022780c2ba8c" is_pre_election: true
I20260619 06:48:06.717486 32674 raft_consensus.cc:2781] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:48:06.717522 32470 raft_consensus.cc:2454] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6945f02f2d54862aad0f78478f84b55 in term 0.
I20260619 06:48:06.717651 32674 raft_consensus.cc:482] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:48:06.717758 32674 raft_consensus.cc:3037] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:48:06.719990 32674 raft_consensus.cc:504] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } }
I20260619 06:48:06.720605 32674 leader_election.cc:288] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [CANDIDATE]: Term 1 election: Requested vote from peers cbdcc43755ca49d89776a664b86582b5 (127.16.79.65:42187), 4e6d13dc4160496bb87b022780c2ba8c (127.16.79.66:43495)
I20260619 06:48:06.721207 32470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e81d2483c224b2980f127a4dec4ad24" candidate_uuid: "d6945f02f2d54862aad0f78478f84b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e6d13dc4160496bb87b022780c2ba8c"
I20260619 06:48:06.721207 32336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e81d2483c224b2980f127a4dec4ad24" candidate_uuid: "d6945f02f2d54862aad0f78478f84b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbdcc43755ca49d89776a664b86582b5"
I20260619 06:48:06.721426 32336 raft_consensus.cc:3037] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:48:06.721426 32470 raft_consensus.cc:3037] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:48:06.724711 32336 raft_consensus.cc:2454] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6945f02f2d54862aad0f78478f84b55 in term 1.
I20260619 06:48:06.724711 32470 raft_consensus.cc:2454] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6945f02f2d54862aad0f78478f84b55 in term 1.
I20260619 06:48:06.725129 32540 leader_election.cc:302] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e6d13dc4160496bb87b022780c2ba8c, d6945f02f2d54862aad0f78478f84b55; no voters:
I20260619 06:48:06.725428 32674 raft_consensus.cc:2781] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:48:06.725914 32674 raft_consensus.cc:686] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [term 1 LEADER]: Becoming Leader. State: Replica: d6945f02f2d54862aad0f78478f84b55, State: Running, Role: LEADER
I20260619 06:48:06.726383 32674 consensus_queue.cc:237] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } }
I20260619 06:48:06.731228 32190 catalog_manager.cc:5495] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6945f02f2d54862aad0f78478f84b55 (127.16.79.67). New cstate: current_term: 1 leader_uuid: "d6945f02f2d54862aad0f78478f84b55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } health_report { overall_health: HEALTHY } } }
I20260619 06:48:06.818099 32665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:06.812581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43960 (local address 127.16.79.67:40643)
0619 06:48:06.812775 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:06.812780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:06.812798 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:48:06.812986 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:06.812990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:06.813075 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:06.813205 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:48:06.813629 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.814347 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.815831 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.816314 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.816409 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:06.816939 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:48:06.817471 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:48:06.817582 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:06.817720 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 06:48:06.817786 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 06:48:06.827049 32336 raft_consensus.cc:1261] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5 [term 1 FOLLOWER]: Refusing update from remote peer d6945f02f2d54862aad0f78478f84b55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:48:06.827047 32470 raft_consensus.cc:1261] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c [term 1 FOLLOWER]: Refusing update from remote peer d6945f02f2d54862aad0f78478f84b55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:48:06.828118 32677 consensus_queue.cc:1035] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:48:06.828529 32674 consensus_queue.cc:1035] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:48:06.842759 32682 mvcc.cc:205] Tried to move back new op lower bound from 7298464509232193536 to 7298464508835004416. Current Snapshot: MvccSnapshot[applied={T|T < 7298464509232193536}]
I20260619 06:48:06.855854 32190 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:48:06.856205 32190 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:48:06.860371 32190 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Sending DeleteTablet for 3 replicas of tablet 6e81d2483c224b2980f127a4dec4ad24
I20260619 06:48:06.861500 32316 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e81d2483c224b2980f127a4dec4ad24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:48:06 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:40010
I20260619 06:48:06.861850 32450 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e81d2483c224b2980f127a4dec4ad24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:48:06 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:54702
I20260619 06:48:06.862545 32690 tablet_replica.cc:330] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5: stopping tablet replica
I20260619 06:48:06.862666 32584 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e81d2483c224b2980f127a4dec4ad24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:48:06 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:43958
I20260619 06:48:06.863135 32690 raft_consensus.cc:2229] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:48:06.863268 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 32253
I20260619 06:48:06.863833 32690 raft_consensus.cc:2258] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:48:06.866083 32690 ts_tablet_manager.cc:1905] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:48:06.873184 32691 tablet_replica.cc:330] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c: stopping tablet replica
I20260619 06:48:06.873698 32691 raft_consensus.cc:2229] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:48:06.874244 32691 raft_consensus.cc:2258] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:48:06.875944 32690 ts_tablet_manager.cc:1918] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:48:06.876175 32690 log.cc:1196] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/6e81d2483c224b2980f127a4dec4ad24
I20260619 06:48:06.876881 32690 ts_tablet_manager.cc:1939] T 6e81d2483c224b2980f127a4dec4ad24 P cbdcc43755ca49d89776a664b86582b5: Deleting consensus metadata
I20260619 06:48:06.879079 32179 catalog_manager.cc:4841] TS cbdcc43755ca49d89776a664b86582b5 (127.16.79.65:42187): tablet 6e81d2483c224b2980f127a4dec4ad24 (table test-table [id=9a294cdd69254b2aba31fb17d682e71b]) successfully deleted
I20260619 06:48:06.879772 32691 ts_tablet_manager.cc:1905] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:48:06.880517 32692 tablet_replica.cc:330] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55: stopping tablet replica
I20260619 06:48:06.881045 32692 raft_consensus.cc:2229] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:48:06.881776 32692 raft_consensus.cc:2258] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:48:06.883994 32692 ts_tablet_manager.cc:1905] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 19 06:48:06 dist-test-slave-5w2x krb5kdc[32147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:48:06.887018 32691 ts_tablet_manager.cc:1918] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:48:06.887192 32691 log.cc:1196] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/6e81d2483c224b2980f127a4dec4ad24
I20260619 06:48:06.887698 32691 ts_tablet_manager.cc:1939] T 6e81d2483c224b2980f127a4dec4ad24 P 4e6d13dc4160496bb87b022780c2ba8c: Deleting consensus metadata
I20260619 06:48:06.889683 32179 catalog_manager.cc:4841] TS 4e6d13dc4160496bb87b022780c2ba8c (127.16.79.66:43495): tablet 6e81d2483c224b2980f127a4dec4ad24 (table test-table [id=9a294cdd69254b2aba31fb17d682e71b]) successfully deleted
I20260619 06:48:06.891407 32692 ts_tablet_manager.cc:1918] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:48:06.891592 32692 log.cc:1196] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/6e81d2483c224b2980f127a4dec4ad24
I20260619 06:48:06.892196 32692 ts_tablet_manager.cc:1939] T 6e81d2483c224b2980f127a4dec4ad24 P d6945f02f2d54862aad0f78478f84b55: Deleting consensus metadata
I20260619 06:48:06.893916 32176 catalog_manager.cc:4841] TS d6945f02f2d54862aad0f78478f84b55 (127.16.79.67:40643): tablet 6e81d2483c224b2980f127a4dec4ad24 (table test-table [id=9a294cdd69254b2aba31fb17d682e71b]) successfully deleted
I20260619 06:48:06.895056 32662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:06.864065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40016 (local address 127.16.79.65:42187)
0619 06:48:06.864295 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:06.864303 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:06.873513 (+ 9210us) server_negotiation.cc:407] Connection header received
0619 06:48:06.873849 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:06.873854 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:06.873945 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:06.874021 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:06.876647 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.877836 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.880172 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:06.880835 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:06.880969 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:06.887107 (+ 6138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:06.887124 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:06.887142 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:06.887187 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:06.891262 (+ 4075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:06.892384 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:06.892394 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:06.892398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:06.892595 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:06.893098 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:06.893103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:06.893107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:06.893603 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:06.893908 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:06.894404 (+ 496us) server_negotiation.cc:299] Negotiation successful
0619 06:48:06.894656 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 06:48:06.895701 32376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:48:06.990000 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 32253
I20260619 06:48:07.008208 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 32387
Jun 19 06:48:07 dist-test-slave-5w2x krb5kdc[32147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:48:07.024855 32666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:07.009070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54710 (local address 127.16.79.66:43495)
0619 06:48:07.009247 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:07.009254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:07.009315 (+ 61us) server_negotiation.cc:407] Connection header received
0619 06:48:07.009532 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:07.009538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:07.009619 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:07.009762 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:07.011148 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:07.011972 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:07.013721 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:07.014195 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:07.014336 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:07.018816 (+ 4480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:07.018833 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:07.018849 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:07.018884 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:07.021907 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:07.022659 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:07.022667 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:07.022669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:07.022806 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:07.023218 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:07.023224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:07.023227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:07.023629 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:07.023943 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:07.024256 (+ 313us) server_negotiation.cc:299] Negotiation successful
0619 06:48:07.024482 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:48:07.025511 32510 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:48:07.116197 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 32387
I20260619 06:48:07.133625 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 32521
Jun 19 06:48:07 dist-test-slave-5w2x krb5kdc[32147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:48:07.150435 32665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:07.134790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43964 (local address 127.16.79.67:40643)
0619 06:48:07.134944 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:07.134948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:07.134962 (+ 14us) server_negotiation.cc:407] Connection header received
0619 06:48:07.135019 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:07.135024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:07.135080 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:07.135211 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:07.136717 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:07.137561 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:07.138952 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:07.139409 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:07.139546 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:07.144308 (+ 4762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:07.144320 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:07.144336 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:07.144378 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:07.147466 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:07.148228 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:07.148235 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:07.148238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:07.148407 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:07.148823 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:07.148827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:07.148829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:07.149164 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:07.149513 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:07.149802 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 06:48:07.150049 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 06:48:07.151203 32644 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:48:07.278501 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 32521
I20260619 06:48:07.297842 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 32160
I20260619 06:48:07.299010 32222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:48:07.412199 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 32160
I20260619 06:48:27.433990 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:38789
--webserver_interface=127.16.79.126
--webserver_port=46775
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:38789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:48:27.766381 32698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:48:27.766706 32698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:48:27.766798 32698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:48:27.774138 32698 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 06:48:27.774245 32698 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 06:48:27.774289 32698 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 06:48:27.774333 32698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:48:27.774379 32698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:48:27.774447 32698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:48:27.774492 32698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:48:27.774554 32698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:48:27.784366 32698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:38789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:38789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=46775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:48:27.785669 32698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:48:27.786940 32698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:48:27.809824 32705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:27.809778 32704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:27.809859 32707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:48:27.810735 32698 server_base.cc:1034] running on GCE node
I20260619 06:48:27.811728 32698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:48:27.813431 32698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:48:27.814682 32698 hybrid_clock.cc:648] HybridClock initialized: now 1781851707814599 us; error 64 us; skew 500 ppm
I20260619 06:48:27.815076 32698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:48:27 dist-test-slave-5w2x krb5kdc[32147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851707, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:48:27.822360 32698 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:48:27.824117 32698 webserver.cc:460] Webserver started at http://127.16.79.126:46775/ using document root <none> and password file <none>
I20260619 06:48:27.824992 32698 fs_manager.cc:359] Metadata directory not provided
I20260619 06:48:27.825153 32698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:48:27.831312 32698 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260619 06:48:27.834342 32714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:27.835788 32698 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:48:27.835985 32698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "52da9733a5904a0d874bbdd4c4e3f23e"
format_stamp: "Formatted at 2026-06-19 06:48:04 on dist-test-slave-5w2x"
server_key: "ad5a977593c517ce564cf936b9fbac01"
server_key_iv: "60fddc4ebada38b4adf7bcdc30c16fc3"
server_key_version: "encryptionkey@0"
I20260619 06:48:27.837090 32698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:48:27.861873 32698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:48:27.866151 32698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:48:27.866456 32698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:48:27.885854 32698 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:38789
I20260619 06:48:27.885878 32765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:38789 every 8 connection(s)
I20260619 06:48:27.887442 32698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:48:27.888553 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 32698
I20260619 06:48:27.889600 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:42187
--local_ip_for_outbound_sockets=127.16.79.65
--tserver_master_addrs=127.16.79.126:38789
--webserver_port=44917
--webserver_interface=127.16.79.65
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:48:27.898313 32766 sys_catalog.cc:263] Verifying existing consensus state
I20260619 06:48:27.901961 32766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Bootstrap starting.
I20260619 06:48:27.922559 32766 log.cc:824] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Log is configured to *not* fsync() on all Append() calls
I20260619 06:48:27.934619 32766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:48:27.935171 32766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Bootstrap complete.
I20260619 06:48:27.945097 32766 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } }
I20260619 06:48:27.946048 32766 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52da9733a5904a0d874bbdd4c4e3f23e, State: Initialized, Role: FOLLOWER
I20260619 06:48:27.946750 32766 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } }
I20260619 06:48:27.947003 32766 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:48:27.947144 32766 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:48:27.947326 32766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 1 FOLLOWER]: Advancing to term 2
I20260619 06:48:27.951375 32766 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } }
I20260619 06:48:27.951906 32766 leader_election.cc:302] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52da9733a5904a0d874bbdd4c4e3f23e; no voters:
I20260619 06:48:27.952641 32766 leader_election.cc:288] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 06:48:27.952775 302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 2 FOLLOWER]: Leader election won for term 2
I20260619 06:48:27.953881 302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 2 LEADER]: Becoming Leader. State: Replica: 52da9733a5904a0d874bbdd4c4e3f23e, State: Running, Role: LEADER
I20260619 06:48:27.954432 302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } }
I20260619 06:48:27.955009 32766 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:48:27.957065 304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52da9733a5904a0d874bbdd4c4e3f23e. Latest consensus state: current_term: 2 leader_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } } }
I20260619 06:48:27.957481 304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [sys.catalog]: This master's current role is: LEADER
I20260619 06:48:27.958034 303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } } }
I20260619 06:48:27.958456 303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [sys.catalog]: This master's current role is: LEADER
I20260619 06:48:27.958972 311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:48:27.977878 311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:48:27.979316 311 catalog_manager.cc:1259] Loaded cluster ID: 3a60bbf1772a4935ba299cdc98638fd3
I20260619 06:48:27.979418 311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:48:27.983501 311 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:48:27.985088 311 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Loaded TSK: 0
I20260619 06:48:28.016801 311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Generated new TSK 1
I20260619 06:48:28.017307 311 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 06:48:28.263366 300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:48:28.263715 300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:48:28.263875 300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:48:28.271420 300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:48:28.271595 300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:48:28.271764 300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:48:28.282213 300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:42187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=44917
--tserver_master_addrs=127.16.79.126:38789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:48:28.283654 300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:48:28.285084 300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:48:28.300597 325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:28.300578 328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:28.300804 326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:48:28.302243 300 server_base.cc:1034] running on GCE node
I20260619 06:48:28.303036 300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:48:28.304287 300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:48:28.305541 300 hybrid_clock.cc:648] HybridClock initialized: now 1781851708305467 us; error 60 us; skew 500 ppm
I20260619 06:48:28.305922 300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:48:28 dist-test-slave-5w2x krb5kdc[32147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851708, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:48:28.309861 300 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:48:28.311625 300 webserver.cc:460] Webserver started at http://127.16.79.65:44917/ using document root <none> and password file <none>
I20260619 06:48:28.312579 300 fs_manager.cc:359] Metadata directory not provided
I20260619 06:48:28.312743 300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:48:28.319931 300 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:48:28.323630 335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:28.325481 300 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 06:48:28.325681 300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "cbdcc43755ca49d89776a664b86582b5"
format_stamp: "Formatted at 2026-06-19 06:48:05 on dist-test-slave-5w2x"
server_key: "132a17df790af093de9b09bb6cd82cef"
server_key_iv: "55588e7e7a7a900a37b136174fe28e93"
server_key_version: "encryptionkey@0"
I20260619 06:48:28.327104 300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:48:28.368191 300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:48:28.372936 300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:48:28.373179 300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:48:28.374146 300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:48:28.375519 300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:48:28.375614 300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:28.375689 300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:48:28.375747 300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:28.410079 300 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:42187
I20260619 06:48:28.410189 448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:42187 every 8 connection(s)
I20260619 06:48:28.411600 300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:48:28.418749 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 300
I20260619 06:48:28.419581 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:43495
--local_ip_for_outbound_sockets=127.16.79.66
--tserver_master_addrs=127.16.79.126:38789
--webserver_port=44745
--webserver_interface=127.16.79.66
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:48:28 dist-test-slave-5w2x krb5kdc[32147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851708, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:48:28.437831 451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:28.414670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:38589 (local address 127.16.79.126:38789)
0619 06:48:28.415314 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:28.415326 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:28.416069 (+ 743us) server_negotiation.cc:407] Connection header received
0619 06:48:28.418538 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:28.418560 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:28.419098 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:28.419561 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:28.421195 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:28.423218 (+ 2023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:28.424890 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:28.426029 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:28.426226 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:28.429880 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:28.429888 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:28.429903 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:28.429946 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:28.433038 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:28.433843 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:28.433850 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:28.433855 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:28.434044 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:28.434500 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:28.434503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:28.434505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:28.434987 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:28.435190 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:28.435999 (+ 809us) server_negotiation.cc:299] Negotiation successful
0619 06:48:28.436456 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":318,"threads_started":1}
I20260619 06:48:28.437834 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:28.414962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:38589)
0619 06:48:28.415872 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:28.415925 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 06:48:28.417722 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:48:28.419671 (+ 1949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:48:28.419678 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:48:28.420396 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:48:28.420963 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:28.420982 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:28.423403 (+ 2421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:28.423407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:28.424747 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:28.424759 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:28.426242 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:28.426246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:28.426436 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:28.427189 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:48:28.427213 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:48:28.429567 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:48:28.433211 (+ 3644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:48:28.433220 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:48:28.433237 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:48:28.433685 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:48:28.434166 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:48:28.434171 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:48:28.434174 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:48:28.434365 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:48:28.435127 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:48:28.435134 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:48:28.435490 (+ 356us) client_negotiation.cc:769] Sending connection context
0619 06:48:28.435776 (+ 286us) client_negotiation.cc:240] Negotiation successful
0619 06:48:28.436439 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":368,"threads_started":1}
I20260619 06:48:28.441190 449 heartbeater.cc:347] Connected to a master server at 127.16.79.126:38789
I20260619 06:48:28.441530 449 heartbeater.cc:464] Registering TS with master...
I20260619 06:48:28.442261 449 heartbeater.cc:511] Master 127.16.79.126:38789 requested a full tablet report, sending...
I20260619 06:48:28.444542 32731 ts_manager.cc:194] Registered new tserver with Master: cbdcc43755ca49d89776a664b86582b5 (127.16.79.65:42187)
I20260619 06:48:28.446007 32731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:38589
W20260619 06:48:28.754104 453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:48:28.754449 453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:48:28.754624 453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:48:28.762110 453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:48:28.762286 453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:48:28.762468 453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:48:28.773448 453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:43495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=44745
--tserver_master_addrs=127.16.79.126:38789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:48:28.774780 453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:48:28.776129 453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:48:28.789376 458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:28.789489 459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:28.789440 461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:48:28.793678 453 server_base.cc:1034] running on GCE node
I20260619 06:48:28.794355 453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:48:28.795524 453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:48:28.796783 453 hybrid_clock.cc:648] HybridClock initialized: now 1781851708796710 us; error 61 us; skew 500 ppm
I20260619 06:48:28.797139 453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:48:28 dist-test-slave-5w2x krb5kdc[32147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851708, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:48:28.800382 453 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:48:28.801987 453 webserver.cc:460] Webserver started at http://127.16.79.66:44745/ using document root <none> and password file <none>
I20260619 06:48:28.802704 453 fs_manager.cc:359] Metadata directory not provided
I20260619 06:48:28.802860 453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:48:28.808562 453 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 06:48:28.811280 468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:28.812727 453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:48:28.812906 453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "4e6d13dc4160496bb87b022780c2ba8c"
format_stamp: "Formatted at 2026-06-19 06:48:05 on dist-test-slave-5w2x"
server_key: "1285c49c25acd9f6a5584d201254d13b"
server_key_iv: "d5b2ce0ffbad184be1e515b6db32458d"
server_key_version: "encryptionkey@0"
I20260619 06:48:28.813879 453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:48:28.833161 453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:48:28.837456 453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:48:28.837736 453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:48:28.838734 453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:48:28.840104 453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:48:28.840263 453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260619 06:48:28.840406 453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:48:28.840497 453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:28.874202 453 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:43495
I20260619 06:48:28.874253 580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:43495 every 8 connection(s)
I20260619 06:48:28.875725 453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:48:28.885690 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 453
I20260619 06:48:28.886467 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:40643
--local_ip_for_outbound_sockets=127.16.79.67
--tserver_master_addrs=127.16.79.126:38789
--webserver_port=42645
--webserver_interface=127.16.79.67
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 06:48:28 dist-test-slave-5w2x krb5kdc[32147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851708, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:48:28.896836 451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:28.878208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:43683 (local address 127.16.79.126:38789)
0619 06:48:28.878409 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:28.878417 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:28.879730 (+ 1313us) server_negotiation.cc:407] Connection header received
0619 06:48:28.881776 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:28.881781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:28.881876 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:28.881972 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:28.883571 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:28.884374 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:28.886306 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:28.886923 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:28.887067 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:28.890667 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:28.890672 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:28.890675 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:28.890710 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:28.893373 (+ 2663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:28.894247 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:28.894253 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:28.894258 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:28.894388 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:28.894825 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:28.894832 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:28.894835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:28.895198 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:28.895369 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:28.896212 (+ 843us) server_negotiation.cc:299] Negotiation successful
0619 06:48:28.896492 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 06:48:28.897939 583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:28.878761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:43683)
0619 06:48:28.879551 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:28.879602 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 06:48:28.881459 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:48:28.882211 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:48:28.882219 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:48:28.882785 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:48:28.883373 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:28.883390 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:28.884582 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:28.884586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:28.886125 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:28.886139 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:28.887174 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:28.887182 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:28.887425 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:28.888125 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:48:28.888207 (+ 82us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:48:28.890403 (+ 2196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:48:28.893533 (+ 3130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:48:28.893542 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:48:28.893560 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:48:28.894114 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:48:28.894501 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:48:28.894506 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:48:28.894510 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:48:28.894712 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:48:28.895441 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:48:28.895447 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:48:28.895797 (+ 350us) client_negotiation.cc:769] Sending connection context
0619 06:48:28.896088 (+ 291us) client_negotiation.cc:240] Negotiation successful
0619 06:48:28.896670 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":343,"threads_started":1}
I20260619 06:48:28.899560 581 heartbeater.cc:347] Connected to a master server at 127.16.79.126:38789
I20260619 06:48:28.899902 581 heartbeater.cc:464] Registering TS with master...
I20260619 06:48:28.900761 581 heartbeater.cc:511] Master 127.16.79.126:38789 requested a full tablet report, sending...
I20260619 06:48:28.902251 32731 ts_manager.cc:194] Registered new tserver with Master: 4e6d13dc4160496bb87b022780c2ba8c (127.16.79.66:43495)
I20260619 06:48:28.903091 32731 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:43683
W20260619 06:48:29.230139 584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:48:29.230453 584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:48:29.230551 584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:48:29.238256 584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:48:29.238377 584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:48:29.238500 584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:48:29.248575 584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:40643
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=42645
--tserver_master_addrs=127.16.79.126:38789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:48:29.249871 584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:48:29.251128 584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:48:29.267400 590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:29.267557 592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:29.267810 589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:48:29.269145 584 server_base.cc:1034] running on GCE node
I20260619 06:48:29.269789 584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:48:29.270851 584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:48:29.272105 584 hybrid_clock.cc:648] HybridClock initialized: now 1781851709272051 us; error 40 us; skew 500 ppm
I20260619 06:48:29.272521 584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:48:29 dist-test-slave-5w2x krb5kdc[32147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851709, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:48:29.275794 584 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:48:29.277433 584 webserver.cc:460] Webserver started at http://127.16.79.67:42645/ using document root <none> and password file <none>
I20260619 06:48:29.278194 584 fs_manager.cc:359] Metadata directory not provided
I20260619 06:48:29.278363 584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:48:29.284072 584 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 06:48:29.287096 599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:29.288852 584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:48:29.289052 584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "d6945f02f2d54862aad0f78478f84b55"
format_stamp: "Formatted at 2026-06-19 06:48:06 on dist-test-slave-5w2x"
server_key: "3f63f8f37499d138b59893c8fe224e5a"
server_key_iv: "033b0713d9cd2921ef5c34bd2cd22efc"
server_key_version: "encryptionkey@0"
I20260619 06:48:29.290092 584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:48:29.326751 584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:48:29.331622 584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:48:29.331964 584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:48:29.333297 584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:48:29.334686 584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:48:29.334815 584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:29.334956 584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:48:29.335053 584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:29.370985 584 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:40643
I20260619 06:48:29.371081 711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:40643 every 8 connection(s)
I20260619 06:48:29.372570 584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:48:29.377729 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 584
Jun 19 06:48:29 dist-test-slave-5w2x krb5kdc[32147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851709, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:48:29.395052 715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.378777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51010 (local address 127.16.79.126:38789)
0619 06:48:29.380877 (+ 2100us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:29.380884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:29.380911 (+ 27us) server_negotiation.cc:407] Connection header received
0619 06:48:29.380957 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:29.380961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:29.381016 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:29.381133 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:29.382657 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.383427 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.385219 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.385669 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.385818 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.388810 (+ 2992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:29.388816 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:29.388819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:29.388859 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:29.391795 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:29.392651 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:29.392657 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:29.392667 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:29.392803 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:29.393271 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:29.393276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:29.393278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:29.393587 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:29.393868 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:29.394240 (+ 372us) server_negotiation.cc:299] Negotiation successful
0619 06:48:29.394478 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1961,"thread_start_us":1865,"threads_started":1}
I20260619 06:48:29.395437 451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.375342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:35845 (local address 127.16.79.126:38789)
0619 06:48:29.375544 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:29.375548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:29.376799 (+ 1251us) server_negotiation.cc:407] Connection header received
0619 06:48:29.378732 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:29.378738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:29.378833 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:29.378980 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:29.381321 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.382063 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.383697 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.384194 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.384331 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.388223 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:29.388228 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:29.388231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:29.388263 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:29.391558 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:29.392423 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:29.392428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:29.392434 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:29.392550 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:29.393002 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:29.393007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:29.393010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:29.393280 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:29.393465 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:29.394853 (+ 1388us) server_negotiation.cc:299] Negotiation successful
0619 06:48:29.395089 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:48:29.395907 714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.375797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:35845)
0619 06:48:29.376632 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:29.376671 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 06:48:29.378402 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:48:29.379317 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:48:29.379334 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:48:29.380348 (+ 1014us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:48:29.381126 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:29.381145 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.382251 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.382256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:29.383533 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:29.383545 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.384452 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.384460 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:29.384757 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.385574 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:48:29.385603 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:48:29.387963 (+ 2360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:48:29.391727 (+ 3764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:48:29.391737 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:48:29.391756 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:48:29.392294 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:48:29.392666 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:48:29.392671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:48:29.392674 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:48:29.392869 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:48:29.393436 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:48:29.393445 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:48:29.393862 (+ 417us) client_negotiation.cc:769] Sending connection context
0619 06:48:29.394167 (+ 305us) client_negotiation.cc:240] Negotiation successful
0619 06:48:29.394615 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":353,"threads_started":1}
I20260619 06:48:29.397405 712 heartbeater.cc:347] Connected to a master server at 127.16.79.126:38789
I20260619 06:48:29.397737 712 heartbeater.cc:464] Registering TS with master...
I20260619 06:48:29.398572 712 heartbeater.cc:511] Master 127.16.79.126:38789 requested a full tablet report, sending...
I20260619 06:48:29.400086 32730 ts_manager.cc:194] Registered new tserver with Master: d6945f02f2d54862aad0f78478f84b55 (127.16.79.67:40643)
I20260619 06:48:29.401082 32730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:35845
I20260619 06:48:29.410624 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:48:29.418004 715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.411754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51012 (local address 127.16.79.126:38789)
0619 06:48:29.411998 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:29.412005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:29.412370 (+ 365us) server_negotiation.cc:407] Connection header received
0619 06:48:29.412579 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:29.412584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:29.412783 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:29.412955 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:48:29.413383 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.414526 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.416071 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.416620 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.416720 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.417196 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:48:29.417393 (+ 197us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 06:48:29.417794 (+ 401us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51012: token expired
Metrics: {"server-negotiator.queue_time_us":76}
W20260619 06:48:29.418164 715 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51012: token expired
W20260619 06:48:29.418699 32652 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.16.79.126:38789)
I20260619 06:48:29.432021 715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.419266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51024 (local address 127.16.79.126:38789)
0619 06:48:29.419433 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:29.419438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:29.419530 (+ 92us) server_negotiation.cc:407] Connection header received
0619 06:48:29.419703 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:29.419708 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:29.419776 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:29.419882 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:29.421323 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.422070 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.423510 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.423900 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.424028 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.426708 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:29.426713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:29.426716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:29.426750 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:29.429279 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:29.430011 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:29.430024 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:29.430027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:29.430151 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:29.430575 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:29.430580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:29.430583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:29.430878 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:29.431133 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:29.431447 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 06:48:29.431672 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 06:48:29.435568 32730 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:48:29.444010 32730 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:48:29.449208 449 heartbeater.cc:503] Master 127.16.79.126:38789 was elected leader, sending a full tablet report...
I20260619 06:48:29.473996 723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.457051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:40643 (local address 127.0.0.1:41784)
0619 06:48:29.460329 (+ 3278us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:29.460347 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:48:29.460479 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:48:29.461734 (+ 1255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:48:29.461746 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:48:29.461787 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:48:29.462300 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:29.462316 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.465439 (+ 3123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.465448 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:29.468967 (+ 3519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:29.468994 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.472570 (+ 3576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.472579 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:29.473115 (+ 536us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.473188 (+ 73us) client_negotiation.cc:769] Sending connection context
0619 06:48:29.473340 (+ 152us) client_negotiation.cc:240] Negotiation successful
0619 06:48:29.473556 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3160,"thread_start_us":2975,"threads_started":1}
I20260619 06:48:29.474002 725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.457891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41784 (local address 127.16.79.67:40643)
0619 06:48:29.459035 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:29.459044 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:29.460613 (+ 1569us) server_negotiation.cc:407] Connection header received
0619 06:48:29.460727 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:29.460737 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:29.461108 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:29.461339 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:48:29.462569 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.465191 (+ 2622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.469296 (+ 4105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.471820 (+ 2524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.472036 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.472382 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:29.473420 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0619 06:48:29.473622 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":985,"thread_start_us":1195,"threads_started":1}
I20260619 06:48:29.478564 647 tablet_service.cc:1467] Processing CreateTablet for tablet e1d9677d6f404d3d9c6936bbb0878386 (DEFAULT_TABLE table=test-table [id=14d7297afa524873b5511a2006813d5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:48:29.480329 647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1d9677d6f404d3d9c6936bbb0878386. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:48:29.481197 722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.456806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42187 (local address 127.0.0.1:55456)
0619 06:48:29.458439 (+ 1633us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:29.458498 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 06:48:29.458709 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:48:29.474117 (+ 15408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:48:29.474124 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:48:29.474161 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:48:29.474564 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:29.474576 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.476353 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.476361 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:29.478362 (+ 2001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:29.478379 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.480328 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.480334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:29.480522 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.480560 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 06:48:29.480633 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 06:48:29.480773 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1311,"mutex_wait_us":45,"thread_start_us":1104,"threads_started":1}
I20260619 06:48:29.481921 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.460104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:43495 (local address 127.0.0.1:50062)
0619 06:48:29.473233 (+ 13129us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:29.473250 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 06:48:29.473420 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:48:29.474885 (+ 1465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:48:29.474889 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:48:29.474915 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:48:29.475274 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:29.475284 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.477083 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.477088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:29.478956 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:29.478967 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.481003 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.481012 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:29.481229 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.481261 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 06:48:29.481348 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 06:48:29.481492 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13011,"mutex_wait_us":1,"thread_start_us":11373,"threads_started":1}
I20260619 06:48:29.482072 724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.457207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55456 (local address 127.16.79.65:42187)
0619 06:48:29.473143 (+ 15936us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:29.473153 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:29.473184 (+ 31us) server_negotiation.cc:407] Connection header received
0619 06:48:29.473306 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:29.473317 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:29.473618 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:29.473854 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:48:29.474768 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.476221 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.478559 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.480038 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.481207 (+ 1169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.481466 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:29.481543 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 06:48:29.481728 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15745,"thread_start_us":15582,"threads_started":1}
I20260619 06:48:29.482167 726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.460611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50062 (local address 127.16.79.66:43495)
0619 06:48:29.472687 (+ 12076us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:29.472700 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:29.474308 (+ 1608us) server_negotiation.cc:407] Connection header received
0619 06:48:29.474415 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:29.474423 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:29.474698 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:29.474903 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:48:29.475434 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.476923 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.479113 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.480474 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.480608 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.480809 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:29.481710 (+ 901us) server_negotiation.cc:299] Negotiation successful
0619 06:48:29.481886 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11835,"thread_start_us":11695,"threads_started":1}
I20260619 06:48:29.485503 516 tablet_service.cc:1467] Processing CreateTablet for tablet e1d9677d6f404d3d9c6936bbb0878386 (DEFAULT_TABLE table=test-table [id=14d7297afa524873b5511a2006813d5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:48:29.485926 383 tablet_service.cc:1467] Processing CreateTablet for tablet e1d9677d6f404d3d9c6936bbb0878386 (DEFAULT_TABLE table=test-table [id=14d7297afa524873b5511a2006813d5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:48:29.486831 516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1d9677d6f404d3d9c6936bbb0878386. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:48:29.487447 383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1d9677d6f404d3d9c6936bbb0878386. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:48:29.498435 728 tablet_bootstrap.cc:492] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55: Bootstrap starting.
I20260619 06:48:29.502719 730 tablet_bootstrap.cc:492] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5: Bootstrap starting.
I20260619 06:48:29.503216 728 tablet_bootstrap.cc:654] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55: Neither blocks nor log segments found. Creating new log.
I20260619 06:48:29.504086 729 tablet_bootstrap.cc:492] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c: Bootstrap starting.
I20260619 06:48:29.504676 728 log.cc:824] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55: Log is configured to *not* fsync() on all Append() calls
I20260619 06:48:29.506879 730 tablet_bootstrap.cc:654] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5: Neither blocks nor log segments found. Creating new log.
I20260619 06:48:29.507813 729 tablet_bootstrap.cc:654] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c: Neither blocks nor log segments found. Creating new log.
I20260619 06:48:29.508283 730 log.cc:824] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5: Log is configured to *not* fsync() on all Append() calls
I20260619 06:48:29.508600 728 tablet_bootstrap.cc:492] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55: No bootstrap required, opened a new log
I20260619 06:48:29.508977 728 ts_tablet_manager.cc:1397] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260619 06:48:29.509217 729 log.cc:824] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c: Log is configured to *not* fsync() on all Append() calls
I20260619 06:48:29.512221 730 tablet_bootstrap.cc:492] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5: No bootstrap required, opened a new log
I20260619 06:48:29.512540 730 ts_tablet_manager.cc:1397] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260619 06:48:29.514997 729 tablet_bootstrap.cc:492] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c: No bootstrap required, opened a new log
I20260619 06:48:29.515282 729 ts_tablet_manager.cc:1397] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260619 06:48:29.517156 728 raft_consensus.cc:348] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } }
I20260619 06:48:29.517604 728 raft_consensus.cc:374] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:48:29.517711 728 raft_consensus.cc:729] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6945f02f2d54862aad0f78478f84b55, State: Initialized, Role: FOLLOWER
I20260619 06:48:29.518534 728 consensus_queue.cc:260] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } }
I20260619 06:48:29.519678 712 heartbeater.cc:503] Master 127.16.79.126:38789 was elected leader, sending a full tablet report...
I20260619 06:48:29.520452 728 ts_tablet_manager.cc:1428] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260619 06:48:29.523365 730 raft_consensus.cc:348] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } }
I20260619 06:48:29.523883 730 raft_consensus.cc:374] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:48:29.524008 730 raft_consensus.cc:729] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbdcc43755ca49d89776a664b86582b5, State: Initialized, Role: FOLLOWER
I20260619 06:48:29.524678 730 consensus_queue.cc:260] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } }
I20260619 06:48:29.526046 729 raft_consensus.cc:348] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } }
I20260619 06:48:29.526544 729 raft_consensus.cc:374] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:48:29.526590 730 ts_tablet_manager.cc:1428] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260619 06:48:29.526667 729 raft_consensus.cc:729] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e6d13dc4160496bb87b022780c2ba8c, State: Initialized, Role: FOLLOWER
I20260619 06:48:29.527281 729 consensus_queue.cc:260] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } }
I20260619 06:48:29.528414 581 heartbeater.cc:503] Master 127.16.79.126:38789 was elected leader, sending a full tablet report...
I20260619 06:48:29.529035 729 ts_tablet_manager.cc:1428] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c: Time spent starting tablet: real 0.014s user 0.013s sys 0.002s
W20260619 06:48:29.624303 713 tablet.cc:2369] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:48:29.627838 582 tablet.cc:2369] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:48:29.664223 450 tablet.cc:2369] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:48:29.686671 735 raft_consensus.cc:482] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:48:29.686975 735 raft_consensus.cc:504] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } }
I20260619 06:48:29.688820 735 leader_election.cc:288] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6945f02f2d54862aad0f78478f84b55 (127.16.79.67:40643), 4e6d13dc4160496bb87b022780c2ba8c (127.16.79.66:43495)
I20260619 06:48:29.696386 725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.689155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:57093 (local address 127.16.79.67:40643)
0619 06:48:29.689429 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:29.689438 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:29.690181 (+ 743us) server_negotiation.cc:407] Connection header received
0619 06:48:29.690379 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:29.690386 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:29.690468 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:29.690560 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:48:29.691242 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.692071 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.694107 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.695140 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.695321 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.695395 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:29.695979 (+ 584us) server_negotiation.cc:299] Negotiation successful
0619 06:48:29.696102 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260619 06:48:29.696509 737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.689035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:40643 (local address 127.16.79.65:57093)
0619 06:48:29.690016 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:29.690043 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 06:48:29.690227 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:48:29.690593 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:48:29.690598 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:48:29.690621 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:48:29.691079 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:29.691091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.692264 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.692272 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:29.693955 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:29.693968 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.695586 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.695592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:29.695766 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.695800 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 06:48:29.695916 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 06:48:29.696071 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"mutex_wait_us":27,"thread_start_us":598,"threads_started":1}
I20260619 06:48:29.696755 726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.689579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:42045 (local address 127.16.79.66:43495)
0619 06:48:29.689780 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:29.689786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:29.690383 (+ 597us) server_negotiation.cc:407] Connection header received
0619 06:48:29.690568 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:29.690573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:29.690652 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:29.690762 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:48:29.691503 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.692319 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.694289 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.695561 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.695715 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.695765 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:29.696340 (+ 575us) server_negotiation.cc:299] Negotiation successful
0619 06:48:29.696486 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 06:48:29.697335 738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.689304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:43495 (local address 127.16.79.65:42045)
0619 06:48:29.690240 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:29.690254 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 06:48:29.690402 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:48:29.690794 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:48:29.690797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:48:29.690820 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:48:29.691307 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:29.691321 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.692477 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.692481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:29.694053 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:48:29.694069 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.695928 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.695934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:48:29.696108 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.696181 (+ 73us) client_negotiation.cc:769] Sending connection context
0619 06:48:29.696843 (+ 662us) client_negotiation.cc:240] Negotiation successful
0619 06:48:29.696992 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"mutex_wait_us":52,"thread_start_us":628,"threads_started":1}
I20260619 06:48:29.697557 667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d9677d6f404d3d9c6936bbb0878386" candidate_uuid: "cbdcc43755ca49d89776a664b86582b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6945f02f2d54862aad0f78478f84b55" is_pre_election: true
I20260619 06:48:29.698150 667 raft_consensus.cc:2454] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbdcc43755ca49d89776a664b86582b5 in term 0.
I20260619 06:48:29.698382 536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d9677d6f404d3d9c6936bbb0878386" candidate_uuid: "cbdcc43755ca49d89776a664b86582b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e6d13dc4160496bb87b022780c2ba8c" is_pre_election: true
I20260619 06:48:29.698865 536 raft_consensus.cc:2454] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbdcc43755ca49d89776a664b86582b5 in term 0.
I20260619 06:48:29.698895 336 leader_election.cc:302] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbdcc43755ca49d89776a664b86582b5, d6945f02f2d54862aad0f78478f84b55; no voters:
I20260619 06:48:29.699272 735 raft_consensus.cc:2781] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:48:29.699458 735 raft_consensus.cc:482] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:48:29.699579 735 raft_consensus.cc:3037] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:48:29.701948 735 raft_consensus.cc:504] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } }
I20260619 06:48:29.702665 735 leader_election.cc:288] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [CANDIDATE]: Term 1 election: Requested vote from peers d6945f02f2d54862aad0f78478f84b55 (127.16.79.67:40643), 4e6d13dc4160496bb87b022780c2ba8c (127.16.79.66:43495)
I20260619 06:48:29.703159 667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d9677d6f404d3d9c6936bbb0878386" candidate_uuid: "cbdcc43755ca49d89776a664b86582b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6945f02f2d54862aad0f78478f84b55"
I20260619 06:48:29.703233 536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1d9677d6f404d3d9c6936bbb0878386" candidate_uuid: "cbdcc43755ca49d89776a664b86582b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e6d13dc4160496bb87b022780c2ba8c"
I20260619 06:48:29.703399 667 raft_consensus.cc:3037] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:48:29.703493 536 raft_consensus.cc:3037] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:48:29.705809 667 raft_consensus.cc:2454] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbdcc43755ca49d89776a664b86582b5 in term 1.
I20260619 06:48:29.705821 536 raft_consensus.cc:2454] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbdcc43755ca49d89776a664b86582b5 in term 1.
I20260619 06:48:29.706352 336 leader_election.cc:302] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbdcc43755ca49d89776a664b86582b5, d6945f02f2d54862aad0f78478f84b55; no voters:
I20260619 06:48:29.706707 735 raft_consensus.cc:2781] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:48:29.707187 735 raft_consensus.cc:686] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [term 1 LEADER]: Becoming Leader. State: Replica: cbdcc43755ca49d89776a664b86582b5, State: Running, Role: LEADER
I20260619 06:48:29.707772 735 consensus_queue.cc:237] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } }
I20260619 06:48:29.712534 32731 catalog_manager.cc:5495] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to cbdcc43755ca49d89776a664b86582b5 (127.16.79.65). New cstate: current_term: 1 leader_uuid: "cbdcc43755ca49d89776a664b86582b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbdcc43755ca49d89776a664b86582b5" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 42187 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 } health_report { overall_health: UNKNOWN } } }
I20260619 06:48:29.766115 724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.760737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55472 (local address 127.16.79.65:42187)
0619 06:48:29.760959 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:29.760963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:29.760984 (+ 21us) server_negotiation.cc:407] Connection header received
0619 06:48:29.761193 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:29.761196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:29.761279 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:29.761435 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:48:29.761811 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.762588 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.763980 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.764451 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.764549 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.765030 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:48:29.765595 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:48:29.765707 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:29.765816 (+ 109us) server_negotiation.cc:299] Negotiation successful
0619 06:48:29.765887 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 06:48:29.774696 536 raft_consensus.cc:1261] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c [term 1 FOLLOWER]: Refusing update from remote peer cbdcc43755ca49d89776a664b86582b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:48:29.774696 667 raft_consensus.cc:1261] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55 [term 1 FOLLOWER]: Refusing update from remote peer cbdcc43755ca49d89776a664b86582b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:48:29.775756 739 consensus_queue.cc:1035] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6945f02f2d54862aad0f78478f84b55" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 40643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:48:29.776177 735 consensus_queue.cc:1035] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e6d13dc4160496bb87b022780c2ba8c" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 43495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:48:29.802594 32731 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:48:29.802922 32731 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:48:29.806818 32731 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Sending DeleteTablet for 3 replicas of tablet e1d9677d6f404d3d9c6936bbb0878386
I20260619 06:48:29.808480 647 tablet_service.cc:1514] Processing DeleteTablet for tablet e1d9677d6f404d3d9c6936bbb0878386 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:48:29 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:41784
I20260619 06:48:29.808527 516 tablet_service.cc:1514] Processing DeleteTablet for tablet e1d9677d6f404d3d9c6936bbb0878386 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:48:29 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:50062
I20260619 06:48:29.808846 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 300
I20260619 06:48:29.809301 383 tablet_service.cc:1514] Processing DeleteTablet for tablet e1d9677d6f404d3d9c6936bbb0878386 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:48:29 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:55456
I20260619 06:48:29.813473 753 tablet_replica.cc:330] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c: stopping tablet replica
I20260619 06:48:29.814116 752 tablet_replica.cc:330] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55: stopping tablet replica
I20260619 06:48:29.814410 754 tablet_replica.cc:330] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5: stopping tablet replica
I20260619 06:48:29.814904 753 raft_consensus.cc:2229] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:48:29.814930 754 raft_consensus.cc:2229] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:48:29.815534 753 raft_consensus.cc:2258] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:48:29.815604 754 raft_consensus.cc:2258] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:48:29.816670 752 raft_consensus.cc:2229] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:48:29.817912 752 raft_consensus.cc:2258] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:48:29.819687 753 ts_tablet_manager.cc:1905] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:48:29.820185 752 ts_tablet_manager.cc:1905] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:48:29.820276 754 ts_tablet_manager.cc:1905] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:48:29.827555 753 ts_tablet_manager.cc:1918] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:48:29.827646 752 ts_tablet_manager.cc:1918] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:48:29.827899 752 log.cc:1196] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/e1d9677d6f404d3d9c6936bbb0878386
I20260619 06:48:29.828598 752 ts_tablet_manager.cc:1939] T e1d9677d6f404d3d9c6936bbb0878386 P d6945f02f2d54862aad0f78478f84b55: Deleting consensus metadata
I20260619 06:48:29.829063 724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.810164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55480 (local address 127.16.79.65:42187)
0619 06:48:29.810342 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:29.810348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:29.810368 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:48:29.810428 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:29.810433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:29.810507 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:29.810638 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:29.812884 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.814165 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.816173 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.816934 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.817102 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.820758 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:29.820778 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:29.820795 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:29.820852 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:29.825046 (+ 4194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:29.826100 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:29.826110 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:29.826114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:29.826296 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:29.826808 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:29.826814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:29.826817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:29.827280 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:29.827613 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:29.828345 (+ 732us) server_negotiation.cc:299] Negotiation successful
0619 06:48:29.828638 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:48:29.830677 444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:48:29.830631 32715 catalog_manager.cc:4841] TS d6945f02f2d54862aad0f78478f84b55 (127.16.79.67:40643): tablet e1d9677d6f404d3d9c6936bbb0878386 (table test-table [id=14d7297afa524873b5511a2006813d5c]) successfully deleted
I20260619 06:48:29.834618 753 log.cc:1196] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/e1d9677d6f404d3d9c6936bbb0878386
I20260619 06:48:29.835245 753 ts_tablet_manager.cc:1939] T e1d9677d6f404d3d9c6936bbb0878386 P 4e6d13dc4160496bb87b022780c2ba8c: Deleting consensus metadata
I20260619 06:48:29.837278 32718 catalog_manager.cc:4841] TS 4e6d13dc4160496bb87b022780c2ba8c (127.16.79.66:43495): tablet e1d9677d6f404d3d9c6936bbb0878386 (table test-table [id=14d7297afa524873b5511a2006813d5c]) successfully deleted
I20260619 06:48:29.936254 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 300
I20260619 06:48:29.954963 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 453
W20260619 06:48:29.955054 32718 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.79.65:42187 (error 108)
I20260619 06:48:29.957022 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.956623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42187 (local address 127.0.0.1:55490)
0619 06:48:29.956787 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:29.956885 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260619 06:48:29.957240 32718 catalog_manager.cc:4584] TS cbdcc43755ca49d89776a664b86582b5 (127.16.79.65:42187): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e1d9677d6f404d3d9c6936bbb0878386: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42187: connect: Connection refused (error 111)
I20260619 06:48:29.974196 726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:29.955790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50064 (local address 127.16.79.66:43495)
0619 06:48:29.956071 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:29.956077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:29.956097 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:48:29.956386 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:29.956392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:29.956489 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:29.956599 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:29.958283 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.959182 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.960810 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:29.961213 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:29.961349 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:29.968260 (+ 6911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:29.968285 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:29.968302 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:29.968378 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:29.971177 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:29.971980 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:29.971988 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:29.971990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:29.972115 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:29.972572 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:29.972577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:29.972581 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:29.972978 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:29.973286 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:29.973611 (+ 325us) server_negotiation.cc:299] Negotiation successful
0619 06:48:29.973846 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260619 06:48:29.974828 576 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:48:30.010838 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:30.010361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42187 (local address 127.0.0.1:55492)
0619 06:48:30.010597 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:30.010694 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260619 06:48:30.012043 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:30.011640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42187 (local address 127.0.0.1:55502)
0619 06:48:30.011829 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:30.011910 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260619 06:48:30.067803 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 453
I20260619 06:48:30.072906 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:30.072289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42187 (local address 127.0.0.1:55504)
0619 06:48:30.072541 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:30.072663 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260619 06:48:30.074077 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:30.073640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42187 (local address 127.0.0.1:55514)
0619 06:48:30.073818 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:30.073917 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 06:48:30.085387 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 584
I20260619 06:48:30.100060 725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:30.086333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41790 (local address 127.16.79.67:40643)
0619 06:48:30.086538 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 06:48:30.086543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:48:30.086564 (+ 21us) server_negotiation.cc:407] Connection header received
0619 06:48:30.086723 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:48:30.086727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:48:30.086794 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:48:30.086919 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:48:30.088444 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:30.089167 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:30.090742 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:48:30.091222 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:48:30.091328 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:48:30.094294 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:48:30.094310 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:48:30.094326 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:48:30.094365 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:48:30.097131 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:30.097923 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:30.097934 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:30.097940 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:30.098089 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:48:30.098496 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:48:30.098501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:48:30.098506 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:48:30.098872 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:48:30.099167 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0619 06:48:30.099473 (+ 306us) server_negotiation.cc:299] Negotiation successful
0619 06:48:30.099698 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 06:48:30.100863 707 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:48:30.154794 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:30.154095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42187 (local address 127.0.0.1:55530)
0619 06:48:30.154409 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:30.154534 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260619 06:48:30.156262 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:30.155752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:42187 (local address 127.0.0.1:55546)
0619 06:48:30.155965 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:30.156083 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:42187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260619 06:48:30.192040 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 584
I20260619 06:48:30.209100 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 32698
I20260619 06:48:30.210253 32761 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:48:30.327697 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 32698
I20260619 06:48:50.350072 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:38789
--webserver_interface=127.16.79.126
--webserver_port=46775
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:38789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:48:50.682834 760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:48:50.683212 760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:48:50.683382 760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:48:50.690847 760 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 06:48:50.691004 760 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 06:48:50.691098 760 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 06:48:50.691187 760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:48:50.691285 760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:48:50.691409 760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:48:50.691499 760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:48:50.691579 760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:48:50.701792 760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:38789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:38789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=46775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:48:50.703167 760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:48:50.704602 760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:48:50.717254 765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:50.717296 766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:50.717301 768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:48:50.718201 760 server_base.cc:1034] running on GCE node
I20260619 06:48:50.719028 760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:48:50.720566 760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:48:50.721781 760 hybrid_clock.cc:648] HybridClock initialized: now 1781851730721700 us; error 58 us; skew 500 ppm
I20260619 06:48:50.722127 760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:48:50 dist-test-slave-5w2x krb5kdc[32147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851730, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:48:50.733285 760 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:48:50.734969 760 webserver.cc:460] Webserver started at http://127.16.79.126:46775/ using document root <none> and password file <none>
I20260619 06:48:50.735720 760 fs_manager.cc:359] Metadata directory not provided
I20260619 06:48:50.735855 760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:48:50.741529 760 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 06:48:50.744426 775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:50.745812 760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:48:50.746012 760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "52da9733a5904a0d874bbdd4c4e3f23e"
format_stamp: "Formatted at 2026-06-19 06:48:04 on dist-test-slave-5w2x"
server_key: "ad5a977593c517ce564cf936b9fbac01"
server_key_iv: "60fddc4ebada38b4adf7bcdc30c16fc3"
server_key_version: "encryptionkey@0"
I20260619 06:48:50.746981 760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:48:50.759297 760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:48:50.763648 760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:48:50.763948 760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:48:50.783043 760 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:38789
I20260619 06:48:50.783075 826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:38789 every 8 connection(s)
I20260619 06:48:50.784684 760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:48:50.791566 827 sys_catalog.cc:263] Verifying existing consensus state
I20260619 06:48:50.793759 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 760
I20260619 06:48:50.794708 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:42187
--local_ip_for_outbound_sockets=127.16.79.65
--tserver_master_addrs=127.16.79.126:38789
--webserver_port=44917
--webserver_interface=127.16.79.65
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:48:50.795029 827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Bootstrap starting.
I20260619 06:48:50.815671 827 log.cc:824] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Log is configured to *not* fsync() on all Append() calls
I20260619 06:48:50.832523 827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 06:48:50.833102 827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Bootstrap complete.
I20260619 06:48:50.842423 827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } }
I20260619 06:48:50.843387 827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52da9733a5904a0d874bbdd4c4e3f23e, State: Initialized, Role: FOLLOWER
I20260619 06:48:50.844005 827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } }
I20260619 06:48:50.844281 827 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:48:50.844420 827 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:48:50.844597 827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 2 FOLLOWER]: Advancing to term 3
I20260619 06:48:50.848843 827 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } }
I20260619 06:48:50.849408 827 leader_election.cc:302] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52da9733a5904a0d874bbdd4c4e3f23e; no voters:
I20260619 06:48:50.850160 827 leader_election.cc:288] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 06:48:50.850327 831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 3 FOLLOWER]: Leader election won for term 3
I20260619 06:48:50.851682 831 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [term 3 LEADER]: Becoming Leader. State: Replica: 52da9733a5904a0d874bbdd4c4e3f23e, State: Running, Role: LEADER
I20260619 06:48:50.852325 831 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } }
I20260619 06:48:50.852576 827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:48:50.854805 833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52da9733a5904a0d874bbdd4c4e3f23e. Latest consensus state: current_term: 3 leader_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } } }
I20260619 06:48:50.855167 833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [sys.catalog]: This master's current role is: LEADER
I20260619 06:48:50.855727 832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52da9733a5904a0d874bbdd4c4e3f23e" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 38789 } } }
I20260619 06:48:50.856276 832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e [sys.catalog]: This master's current role is: LEADER
I20260619 06:48:50.857966 838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:48:50.868582 838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:48:50.870741 838 catalog_manager.cc:1259] Loaded cluster ID: 3a60bbf1772a4935ba299cdc98638fd3
I20260619 06:48:50.870954 838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:48:50.876250 838 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:48:50.878548 838 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Loaded TSK: 0
I20260619 06:48:50.878856 838 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 52da9733a5904a0d874bbdd4c4e3f23e: Loaded TSK: 1
F20260619 06:48:50.880396 838 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f85f5a689ad google::LogMessage::Fail() at ??:0
@ 0x7f85f5a6b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f85f5a684de google::LogMessage::Flush() at ??:0
@ 0x7f85f5a69a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f85f97cc7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f85f97bb73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f860b4971c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f860b496783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f860b5222a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f860b5220c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f860b56c5e9 std::function<>::operator()() at ??:0
@ 0x7f860b49acdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f860b498c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f860b520610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f860b520439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f85f86b4345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f85f86b791c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f85f86b7799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f85f8692eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f85fb96c6db start_thread at ??:0
@ 0x7f85f490261f clone at ??:0
W20260619 06:48:51.202571 829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:48:51.202901 829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:48:51.203076 829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:48:51.210693 829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:48:51.210857 829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:48:51.211054 829 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:48:51.221294 829 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:42187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=44917
--tserver_master_addrs=127.16.79.126:38789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:48:51.222647 829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:48:51.223982 829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:48:51.237751 874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:51.237862 875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:51.237774 877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:48:51.238772 829 server_base.cc:1034] running on GCE node
I20260619 06:48:51.239341 829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:48:51.240345 829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:48:51.241564 829 hybrid_clock.cc:648] HybridClock initialized: now 1781851731241518 us; error 31 us; skew 500 ppm
I20260619 06:48:51.241915 829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:48:51 dist-test-slave-5w2x krb5kdc[32147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851731, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:48:51.245085 829 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:48:51.246558 829 webserver.cc:460] Webserver started at http://127.16.79.65:44917/ using document root <none> and password file <none>
I20260619 06:48:51.247212 829 fs_manager.cc:359] Metadata directory not provided
I20260619 06:48:51.247316 829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:48:51.252729 829 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260619 06:48:51.256517 884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:51.258569 829 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260619 06:48:51.258780 829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "cbdcc43755ca49d89776a664b86582b5"
format_stamp: "Formatted at 2026-06-19 06:48:05 on dist-test-slave-5w2x"
server_key: "132a17df790af093de9b09bb6cd82cef"
server_key_iv: "55588e7e7a7a900a37b136174fe28e93"
server_key_version: "encryptionkey@0"
I20260619 06:48:51.259778 829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:48:51.277827 829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:48:51.282126 829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:48:51.282545 829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:48:51.283634 829 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:48:51.285609 891 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260619 06:48:51.290850 891 ts_tablet_manager.cc:1852] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260619 06:48:51.291837 891 ts_tablet_manager.cc:1905] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:48:51.298951 891 ts_tablet_manager.cc:1918] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:48:51.299124 891 log.cc:1196] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/e1d9677d6f404d3d9c6936bbb0878386
I20260619 06:48:51.299544 891 ts_tablet_manager.cc:1939] T e1d9677d6f404d3d9c6936bbb0878386 P cbdcc43755ca49d89776a664b86582b5: Deleting consensus metadata
I20260619 06:48:51.299899 829 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260619 06:48:51.300062 829 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.015s user 0.001s sys 0.000s
I20260619 06:48:51.300237 829 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:48:51.300340 829 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:51.334841 829 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:42187
I20260619 06:48:51.334931 997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:42187 every 8 connection(s)
I20260619 06:48:51.336333 829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:48:51.336897 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 829
I20260619 06:48:51.337772 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:43495
--local_ip_for_outbound_sockets=127.16.79.66
--tserver_master_addrs=127.16.79.126:38789
--webserver_port=44745
--webserver_interface=127.16.79.66
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:48:51.341619 1000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:51.339551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:49967)
0619 06:48:51.340417 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:51.340714 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":348,"threads_started":1}
W20260619 06:48:51.342242 998 heartbeater.cc:644] Failed to heartbeat to 127.16.79.126:38789 (0 consecutive failures): Network error: Failed to ping master at 127.16.79.126:38789: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
I20260619 06:48:51.343053 1000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:51.342731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:48735)
0619 06:48:51.342876 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:51.342950 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 06:48:51.343828 1000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:51.343499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:33469)
0619 06:48:51.343636 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:51.343713 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260619 06:48:51.344223 998 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 06:48:51.669569 1001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:48:51.669926 1001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:48:51.670105 1001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:48:51.677572 1001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:48:51.677762 1001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:48:51.677981 1001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:48:51.688103 1001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:43495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=44745
--tserver_master_addrs=127.16.79.126:38789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:48:51.689576 1001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:48:51.690965 1001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:48:51.705436 1009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:51.705451 1006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:51.705461 1007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:48:51.709671 1001 server_base.cc:1034] running on GCE node
I20260619 06:48:51.710378 1001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:48:51.711462 1001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:48:51.712698 1001 hybrid_clock.cc:648] HybridClock initialized: now 1781851731712627 us; error 68 us; skew 500 ppm
I20260619 06:48:51.713040 1001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:48:51 dist-test-slave-5w2x krb5kdc[32147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851731, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:48:51.716496 1001 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:48:51.718138 1001 webserver.cc:460] Webserver started at http://127.16.79.66:44745/ using document root <none> and password file <none>
I20260619 06:48:51.718926 1001 fs_manager.cc:359] Metadata directory not provided
I20260619 06:48:51.719089 1001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:48:51.724917 1001 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 06:48:51.727850 1016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:51.729399 1001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 06:48:51.729601 1001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "4e6d13dc4160496bb87b022780c2ba8c"
format_stamp: "Formatted at 2026-06-19 06:48:05 on dist-test-slave-5w2x"
server_key: "1285c49c25acd9f6a5584d201254d13b"
server_key_iv: "d5b2ce0ffbad184be1e515b6db32458d"
server_key_version: "encryptionkey@0"
I20260619 06:48:51.730650 1001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:48:51.753347 1001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:48:51.758270 1001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:48:51.758582 1001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:48:51.759687 1001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:48:51.761328 1001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:48:51.761468 1001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:51.761608 1001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:48:51.761703 1001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:51.797626 1001 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:43495
I20260619 06:48:51.797686 1128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:43495 every 8 connection(s)
I20260619 06:48:51.799436 1001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:48:51.801959 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 1001
I20260619 06:48:51.802613 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:40643
--local_ip_for_outbound_sockets=127.16.79.67
--tserver_master_addrs=127.16.79.126:38789
--webserver_port=42645
--webserver_interface=127.16.79.67
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:39549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:48:51.805692 1131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:51.803201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:44573)
0619 06:48:51.804084 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:51.804420 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":310,"threads_started":1}
W20260619 06:48:51.806449 1129 heartbeater.cc:644] Failed to heartbeat to 127.16.79.126:38789 (0 consecutive failures): Network error: Failed to ping master at 127.16.79.126:38789: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
I20260619 06:48:51.807646 1131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:51.807105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:39677)
0619 06:48:51.807386 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:51.807481 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260619 06:48:51.811985 1131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:51.808333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:42967)
0619 06:48:51.811613 (+ 3280us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:51.811732 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3139,"threads_started":1}
W20260619 06:48:51.812409 1129 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 06:48:52.137972 1133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:48:52.138252 1133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:48:52.138408 1133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:48:52.145735 1133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:48:52.145848 1133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:48:52.145961 1133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:48:52.155831 1133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:39549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:40643
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=42645
--tserver_master_addrs=127.16.79.126:38789
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:48:52.157208 1133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:48:52.158509 1133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:48:52.171986 1138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:52.171978 1139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:48:52.172241 1141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:48:52.173015 1133 server_base.cc:1034] running on GCE node
I20260619 06:48:52.173666 1133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:48:52.174669 1133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:48:52.175917 1133 hybrid_clock.cc:648] HybridClock initialized: now 1781851732175862 us; error 49 us; skew 500 ppm
I20260619 06:48:52.176360 1133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:48:52 dist-test-slave-5w2x krb5kdc[32147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851732, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:48:52.179608 1133 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:48:52.181154 1133 webserver.cc:460] Webserver started at http://127.16.79.67:42645/ using document root <none> and password file <none>
I20260619 06:48:52.181842 1133 fs_manager.cc:359] Metadata directory not provided
I20260619 06:48:52.181944 1133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:48:52.187256 1133 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 06:48:52.190035 1148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:52.191427 1133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:48:52.191563 1133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "d6945f02f2d54862aad0f78478f84b55"
format_stamp: "Formatted at 2026-06-19 06:48:06 on dist-test-slave-5w2x"
server_key: "3f63f8f37499d138b59893c8fe224e5a"
server_key_iv: "033b0713d9cd2921ef5c34bd2cd22efc"
server_key_version: "encryptionkey@0"
I20260619 06:48:52.192611 1133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:48:52.212384 1133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:48:52.216343 1133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:48:52.216568 1133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:48:52.217706 1133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:48:52.218977 1133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:48:52.219063 1133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:52.219139 1133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:48:52.219192 1133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:48:52.252705 1133 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:40643
I20260619 06:48:52.252795 1260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:40643 every 8 connection(s)
I20260619 06:48:52.254175 1133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:48:52.259020 1263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:52.257263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:42029)
0619 06:48:52.257975 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:52.258183 (+ 208us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":265,"threads_started":1}
W20260619 06:48:52.259645 1261 heartbeater.cc:644] Failed to heartbeat to 127.16.79.126:38789 (0 consecutive failures): Network error: Failed to ping master at 127.16.79.126:38789: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
I20260619 06:48:52.260519 1263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:52.260185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:57621)
0619 06:48:52.260355 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:52.260420 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 06:48:52.260878 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 1133
I20260619 06:48:52.261475 1263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:52.261151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:55623)
0619 06:48:52.261283 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:52.261367 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 06:48:52.261781 1261 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 06:48:52.345436 1265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:52.344678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:58629)
0619 06:48:52.345109 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:52.345214 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":208,"threads_started":1}
I20260619 06:48:52.813738 1266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:52.812958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:51081)
0619 06:48:52.813399 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:52.813490 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":217,"threads_started":1}
I20260619 06:48:53.263226 1267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:53.262350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:47829)
0619 06:48:53.262905 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:53.263012 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":316,"threads_started":1}
I20260619 06:48:53.347004 1268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:53.346326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:50507)
0619 06:48:53.346768 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:53.346853 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":222,"threads_started":1}
I20260619 06:48:53.815470 1269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:53.814736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:39405)
0619 06:48:53.815166 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:53.815303 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":227,"threads_started":1}
I20260619 06:48:54.265045 1270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:54.264341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:35433)
0619 06:48:54.264788 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:54.264884 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":230,"threads_started":1}
I20260619 06:48:54.348628 1271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:54.347870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:34221)
0619 06:48:54.348359 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:54.348463 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":215,"threads_started":1}
I20260619 06:48:54.817101 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:54.816396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:45185)
0619 06:48:54.816827 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:54.816910 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":227,"threads_started":1}
I20260619 06:48:55.266981 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:55.266252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:51249)
0619 06:48:55.266720 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:55.266806 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":243,"threads_started":1}
I20260619 06:48:55.350176 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:55.349458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:56999)
0619 06:48:55.349930 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:55.350014 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":243,"threads_started":1}
I20260619 06:48:55.818661 1275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:55.817971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:60859)
0619 06:48:55.818410 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:55.818492 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":228,"threads_started":1}
I20260619 06:48:56.268656 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:56.267891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:41747)
0619 06:48:56.268381 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:56.268462 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":250,"threads_started":1}
I20260619 06:48:56.351709 1277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:56.351032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:35265)
0619 06:48:56.351471 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:56.351548 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":212,"threads_started":1}
I20260619 06:48:56.820402 1278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:56.819619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:34447)
0619 06:48:56.820057 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:56.820198 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":242,"threads_started":1}
I20260619 06:48:57.270493 1279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:57.269654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:40303)
0619 06:48:57.270219 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:57.270306 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":292,"threads_started":1}
I20260619 06:48:57.353559 1280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:57.352752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:50875)
0619 06:48:57.353256 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:57.353368 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":254,"threads_started":1}
I20260619 06:48:57.822170 1281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:57.821311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:36381)
0619 06:48:57.821890 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:57.821994 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":344,"threads_started":1}
I20260619 06:48:58.272089 1282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:58.271338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:38559)
0619 06:48:58.271827 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:58.271923 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":250,"threads_started":1}
I20260619 06:48:58.355404 1283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:58.354574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:48679)
0619 06:48:58.355091 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:58.355178 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":229,"threads_started":1}
I20260619 06:48:58.824328 1284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:58.823167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:58879)
0619 06:48:58.823750 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:58.823848 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":249,"threads_started":1}
I20260619 06:48:59.274076 1285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:59.273139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:36663)
0619 06:48:59.273768 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:59.273859 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":290,"threads_started":1}
I20260619 06:48:59.357237 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:59.356444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:59989)
0619 06:48:59.356970 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:59.357065 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":333,"threads_started":1}
I20260619 06:48:59.826280 1287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:48:59.825489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:53917)
0619 06:48:59.826001 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0619 06:48:59.826094 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":279,"threads_started":1}
I20260619 06:49:00.275848 1288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:00.275052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:33875)
0619 06:49:00.275589 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:00.275680 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":317,"threads_started":1}
I20260619 06:49:00.359081 1289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:00.358264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:51495)
0619 06:49:00.358810 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:00.358912 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":276,"threads_started":1}
I20260619 06:49:00.828063 1290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:00.827326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:35367)
0619 06:49:00.827786 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:00.827887 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":225,"threads_started":1}
I20260619 06:49:01.277596 1291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:01.276871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:56809)
0619 06:49:01.277339 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:01.277423 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":222,"threads_started":1}
I20260619 06:49:01.360798 1292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:01.360043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:42493)
0619 06:49:01.360532 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:01.360613 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":295,"threads_started":1}
I20260619 06:49:01.829773 1293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:01.829059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:36867)
0619 06:49:01.829520 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:01.829608 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":234,"threads_started":1}
I20260619 06:49:02.279299 1294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:02.278518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:44047)
0619 06:49:02.279058 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:02.279138 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":311,"threads_started":1}
I20260619 06:49:02.362399 1295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:02.361685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:38901)
0619 06:49:02.362130 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:02.362209 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":244,"threads_started":1}
I20260619 06:49:02.831667 1296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:02.830862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:58271)
0619 06:49:02.831388 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:02.831491 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":284,"threads_started":1}
I20260619 06:49:03.281104 1297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:03.280402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:40571)
0619 06:49:03.280839 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:03.280927 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":211,"threads_started":1}
I20260619 06:49:03.364194 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:03.363409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:58523)
0619 06:49:03.363888 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:03.363973 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":254,"threads_started":1}
I20260619 06:49:03.833424 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:03.832720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:59205)
0619 06:49:03.833167 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:03.833253 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":221,"threads_started":1}
I20260619 06:49:04.282922 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:04.282093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:45217)
0619 06:49:04.282632 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:04.282762 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":271,"threads_started":1}
I20260619 06:49:04.366041 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:04.365230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:37769)
0619 06:49:04.365676 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:04.365821 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":229,"threads_started":1}
I20260619 06:49:04.835140 1302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:04.834345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:57139)
0619 06:49:04.834849 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:04.834948 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":265,"threads_started":1}
I20260619 06:49:05.284754 1303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:05.283869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:45841)
0619 06:49:05.284472 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:05.284565 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":368,"threads_started":1}
I20260619 06:49:05.367654 1304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:05.366933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:38819)
0619 06:49:05.367390 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:05.367473 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":219,"threads_started":1}
I20260619 06:49:05.837082 1305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:05.836108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:55791)
0619 06:49:05.836695 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:05.836829 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":267,"threads_started":1}
I20260619 06:49:06.286576 1306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:06.285818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:50049)
0619 06:49:06.286300 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:06.286391 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":236,"threads_started":1}
I20260619 06:49:06.369771 1307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:06.369007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.65:36357)
0619 06:49:06.369467 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:06.369559 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":228,"threads_started":1}
I20260619 06:49:06.839118 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:06.838289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.66:38707)
0619 06:49:06.838756 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:06.838872 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":241,"threads_started":1}
I20260619 06:49:07.272815 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 829
I20260619 06:49:07.288573 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:07.287791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:38789 (local address 127.16.79.67:38925)
0619 06:49:07.288300 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:07.288391 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.126:38789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":237,"threads_started":1}
I20260619 06:49:07.296324 1309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:07.274011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53886 (local address 127.16.79.65:42187)
0619 06:49:07.274747 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:07.274769 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:07.274822 (+ 53us) server_negotiation.cc:407] Connection header received
0619 06:49:07.276662 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:07.276674 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:07.277134 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:07.277455 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:07.278913 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:07.280305 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:07.281543 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:07.282197 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:07.282343 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:07.289948 (+ 7605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:07.289957 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:07.289975 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:07.290016 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:07.293023 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:07.293699 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:07.293706 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:07.293713 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:07.293885 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:07.294253 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:07.294256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:07.294258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:07.294798 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:07.295027 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:07.295435 (+ 408us) server_negotiation.cc:299] Negotiation successful
0619 06:49:07.295948 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":351,"threads_started":1}
I20260619 06:49:07.297835 993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:07.384750 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 829
I20260619 06:49:07.401919 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 1001
I20260619 06:49:07.420077 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:07.402937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57560 (local address 127.16.79.66:43495)
0619 06:49:07.403477 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:07.403491 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:07.403528 (+ 37us) server_negotiation.cc:407] Connection header received
0619 06:49:07.405244 (+ 1716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:07.405254 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:07.405691 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:07.405989 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:07.407336 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:07.408719 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:07.410033 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:07.410728 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:07.410876 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:07.413748 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:07.413756 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:07.413773 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:07.413830 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:07.417004 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:07.417705 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:07.417711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:07.417717 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:07.417890 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:07.418268 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:07.418272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:07.418273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:07.418821 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:07.419047 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:07.419460 (+ 413us) server_negotiation.cc:299] Negotiation successful
0619 06:49:07.419794 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":279,"threads_started":1}
I20260619 06:49:07.421612 1124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:07.506692 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 1001
I20260619 06:49:07.524469 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 1133
I20260619 06:49:07.543205 1315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:07.525606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55026 (local address 127.16.79.67:40643)
0619 06:49:07.526263 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:07.526280 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:07.526326 (+ 46us) server_negotiation.cc:407] Connection header received
0619 06:49:07.527984 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:07.527994 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:07.528546 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:07.528924 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:07.530494 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:07.531726 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:07.533123 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:07.533831 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:07.534059 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:07.536735 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:07.536743 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:07.536760 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:07.536811 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:07.539710 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:07.540561 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:07.540569 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:07.540576 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:07.540746 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:07.541166 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:07.541171 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:07.541173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:07.541704 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:07.542041 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:07.542476 (+ 435us) server_negotiation.cc:299] Negotiation successful
0619 06:49:07.542902 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":316,"threads_started":1}
I20260619 06:49:07.544872 1256 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:07.631718 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 1133
2026-06-19T06:49:07Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64814 ms)
[----------] 30 tests from SecurityITest (296198 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:07 dist-test-slave-5w2x krb5kdc[1319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:07 dist-test-slave-5w2x krb5kdc[1319](info): set up 2 sockets
Jun 19 06:49:07 dist-test-slave-5w2x krb5kdc[1319](info): commencing operation
krb5kdc: starting...
W20260619 06:49:08.740487 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:08 dist-test-slave-5w2x krb5kdc[1319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:49:08.988584 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:36603
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:45149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:36603
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:09.329069 1332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:09.329478 1332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:09.329651 1332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:09.337241 1332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:09.337431 1332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:09.337554 1332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:09.337661 1332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:09.337756 1332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:09.347944 1332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:45149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:36603
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:36603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:09.349473 1332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:09.350891 1332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:09.363302 1337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:09.363276 1338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:09.363247 1340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:09.363794 1332 server_base.cc:1034] running on GCE node
I20260619 06:49:09.364801 1332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:09.366380 1332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:09.367606 1332 hybrid_clock.cc:648] HybridClock initialized: now 1781851749367543 us; error 52 us; skew 500 ppm
I20260619 06:49:09.367944 1332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:09 dist-test-slave-5w2x krb5kdc[1319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851749, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:09.371325 1332 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:49:09.373255 1332 webserver.cc:460] Webserver started at http://127.16.79.126:45947/ using document root <none> and password file <none>
I20260619 06:49:09.374162 1332 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:09.374311 1332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:09.374643 1332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:09.377877 1332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "da442114001244e0b76a7541f9df9209"
format_stamp: "Formatted at 2026-06-19 06:49:09 on dist-test-slave-5w2x"
server_key: "0b04c3928ed8398a3f43545c066ae34f"
server_key_iv: "4987e98be78daf4ca1a6eccb024a3330"
server_key_version: "encryptionkey@0"
I20260619 06:49:09.378881 1332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "da442114001244e0b76a7541f9df9209"
format_stamp: "Formatted at 2026-06-19 06:49:09 on dist-test-slave-5w2x"
server_key: "0b04c3928ed8398a3f43545c066ae34f"
server_key_iv: "4987e98be78daf4ca1a6eccb024a3330"
server_key_version: "encryptionkey@0"
I20260619 06:49:09.384682 1332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260619 06:49:09.388514 1347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:09.390030 1332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:49:09.390244 1332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "da442114001244e0b76a7541f9df9209"
format_stamp: "Formatted at 2026-06-19 06:49:09 on dist-test-slave-5w2x"
server_key: "0b04c3928ed8398a3f43545c066ae34f"
server_key_iv: "4987e98be78daf4ca1a6eccb024a3330"
server_key_version: "encryptionkey@0"
I20260619 06:49:09.390472 1332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:09.403314 1332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:09.407421 1332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:09.407711 1332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:09.426733 1332 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:36603
I20260619 06:49:09.426764 1398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:36603 every 8 connection(s)
I20260619 06:49:09.428208 1332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:09.432288 1399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:09.432559 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 1332
I20260619 06:49:09.432817 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:09.433650 16701 external_mini_cluster.cc:1428] Setting key 212ee9b8a4f213a015697e762c40c965
I20260619 06:49:09.446996 1399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209: Bootstrap starting.
Jun 19 06:49:09 dist-test-slave-5w2x krb5kdc[1319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:49:09.450683 1399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:09.451787 1399 log.cc:824] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:09.455029 1399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209: No bootstrap required, opened a new log
I20260619 06:49:09.459002 1402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:09.435812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38942 (local address 127.16.79.126:36603)
0619 06:49:09.436680 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:09.436691 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:09.436734 (+ 43us) server_negotiation.cc:407] Connection header received
0619 06:49:09.438855 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:09.438882 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:09.439284 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:09.439644 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:09.441550 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:09.442884 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:09.444847 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:09.445543 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:09.445705 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:09.450515 (+ 4810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:09.450523 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:09.450537 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:09.450577 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:09.454238 (+ 3661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:09.455028 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:09.455036 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:09.455045 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:09.455251 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:09.455685 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:09.455699 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:09.455702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:09.456433 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:09.456745 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:09.457145 (+ 400us) server_negotiation.cc:299] Negotiation successful
0619 06:49:09.457606 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":438,"threads_started":1}
I20260619 06:49:09.464612 1399 raft_consensus.cc:348] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da442114001244e0b76a7541f9df9209" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36603 } }
I20260619 06:49:09.464963 1399 raft_consensus.cc:374] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:09.465035 1399 raft_consensus.cc:729] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da442114001244e0b76a7541f9df9209, State: Initialized, Role: FOLLOWER
I20260619 06:49:09.465538 1399 consensus_queue.cc:260] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da442114001244e0b76a7541f9df9209" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36603 } }
I20260619 06:49:09.465740 1399 raft_consensus.cc:388] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:09.465837 1399 raft_consensus.cc:482] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:09.465951 1399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:09.468096 1399 raft_consensus.cc:504] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da442114001244e0b76a7541f9df9209" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36603 } }
I20260619 06:49:09.468637 1399 leader_election.cc:302] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da442114001244e0b76a7541f9df9209; no voters:
I20260619 06:49:09.469244 1399 leader_election.cc:288] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:09.469414 1404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:09.470559 1404 raft_consensus.cc:686] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [term 1 LEADER]: Becoming Leader. State: Replica: da442114001244e0b76a7541f9df9209, State: Running, Role: LEADER
I20260619 06:49:09.471197 1404 consensus_queue.cc:237] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da442114001244e0b76a7541f9df9209" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36603 } }
I20260619 06:49:09.471542 1399 sys_catalog.cc:564] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:09.473973 1406 sys_catalog.cc:455] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da442114001244e0b76a7541f9df9209. Latest consensus state: current_term: 1 leader_uuid: "da442114001244e0b76a7541f9df9209" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da442114001244e0b76a7541f9df9209" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36603 } } }
I20260619 06:49:09.474315 1406 sys_catalog.cc:458] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:09.474823 1405 sys_catalog.cc:455] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da442114001244e0b76a7541f9df9209" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da442114001244e0b76a7541f9df9209" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 36603 } } }
I20260619 06:49:09.475140 1405 sys_catalog.cc:458] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:09.475368 1411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:09.481230 1411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:09.490411 1411 catalog_manager.cc:1347] Generated new cluster ID: 47d07988cad04d5abcb2edbd3e3fbc8f
I20260619 06:49:09.490535 1411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:09.503371 1411 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:09.504918 1411 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:09.517659 1411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209: Generated new TSK 0
I20260619 06:49:09.518857 1411 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:49:09.584286 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:36603
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:45149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:09.919351 1425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:09.919711 1425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:09.919898 1425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:09.927464 1425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:09.927635 1425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:09.927838 1425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:49:09.937914 1425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:45149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:36603
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:09.939292 1425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:09.940716 1425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:09.954062 1430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:09.954026 1431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:09.953986 1433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:09.954839 1425 server_base.cc:1034] running on GCE node
I20260619 06:49:09.955489 1425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:09.956550 1425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:09.957810 1425 hybrid_clock.cc:648] HybridClock initialized: now 1781851749957733 us; error 37 us; skew 500 ppm
I20260619 06:49:09.958204 1425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:09 dist-test-slave-5w2x krb5kdc[1319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851749, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:09.961838 1425 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:49:09.963395 1425 webserver.cc:460] Webserver started at http://127.16.79.65:44091/ using document root <none> and password file <none>
I20260619 06:49:09.964074 1425 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:09.964265 1425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:09.964556 1425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:09.967504 1425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "0efa6eafbb9a4bffb491d6457a0dffca"
format_stamp: "Formatted at 2026-06-19 06:49:09 on dist-test-slave-5w2x"
server_key: "dfb492104f4287cbbe07917b79e43738"
server_key_iv: "a5180121af21f6ca3d83ced845ee5da7"
server_key_version: "encryptionkey@0"
I20260619 06:49:09.968333 1425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "0efa6eafbb9a4bffb491d6457a0dffca"
format_stamp: "Formatted at 2026-06-19 06:49:09 on dist-test-slave-5w2x"
server_key: "dfb492104f4287cbbe07917b79e43738"
server_key_iv: "a5180121af21f6ca3d83ced845ee5da7"
server_key_version: "encryptionkey@0"
I20260619 06:49:09.973986 1425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:49:09.977795 1440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:09.979409 1425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:49:09.979583 1425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "0efa6eafbb9a4bffb491d6457a0dffca"
format_stamp: "Formatted at 2026-06-19 06:49:09 on dist-test-slave-5w2x"
server_key: "dfb492104f4287cbbe07917b79e43738"
server_key_iv: "a5180121af21f6ca3d83ced845ee5da7"
server_key_version: "encryptionkey@0"
I20260619 06:49:09.979797 1425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:09.999548 1425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:10.003621 1425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:10.003933 1425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:10.005043 1425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:10.006414 1425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:10.006489 1425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:10.006613 1425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:10.006675 1425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:10.039786 1425 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:37033
I20260619 06:49:10.039851 1552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:37033 every 8 connection(s)
I20260619 06:49:10.041258 1425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:49:10.041718 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 1425
I20260619 06:49:10.041981 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:49:10.042775 16701 external_mini_cluster.cc:1428] Setting key f59eb83a6568ade1942dbb5153ce1d12
Jun 19 06:49:10 dist-test-slave-5w2x krb5kdc[1319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851749, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:49:10.066828 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:10.044278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:46703 (local address 127.16.79.126:36603)
0619 06:49:10.044794 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:10.044802 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:10.046034 (+ 1232us) server_negotiation.cc:407] Connection header received
0619 06:49:10.048983 (+ 2949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:10.048988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:10.049055 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:10.049152 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:10.051313 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:10.052035 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:10.054527 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:10.055241 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:10.055434 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:10.059382 (+ 3948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:10.059391 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:10.059399 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:10.059445 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:10.063209 (+ 3764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:10.064025 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:10.064038 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:10.064045 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:10.064229 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:10.064702 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:10.064710 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:10.064716 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:10.065149 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:10.065325 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:10.066064 (+ 739us) server_negotiation.cc:299] Negotiation successful
0619 06:49:10.066355 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":245,"threads_started":1}
I20260619 06:49:10.067639 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:10.044999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36603 (local address 127.16.79.65:46703)
0619 06:49:10.045867 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:10.045915 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:10.048670 (+ 2755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:10.049443 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:10.049457 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:10.050263 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:10.051108 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:10.051138 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:10.052230 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:10.052239 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:10.054382 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:10.054394 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:10.055500 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:10.055505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:10.055713 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:10.056530 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:49:10.056552 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:10.059131 (+ 2579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:10.063389 (+ 4258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:10.063396 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:10.063410 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:10.063876 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:10.064355 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:10.064358 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:10.064361 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:10.064563 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:10.065304 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:10.065310 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:10.065658 (+ 348us) client_negotiation.cc:769] Sending connection context
0619 06:49:10.065895 (+ 237us) client_negotiation.cc:240] Negotiation successful
0619 06:49:10.066434 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":300,"threads_started":1}
I20260619 06:49:10.069329 1553 heartbeater.cc:347] Connected to a master server at 127.16.79.126:36603
I20260619 06:49:10.069671 1553 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
I20260619 06:49:10.070451 1553 heartbeater.cc:511] Master 127.16.79.126:36603 requested a full tablet report, sending...
I20260619 06:49:10.072893 1364 ts_manager.cc:194] Registered new tserver with Master: 0efa6eafbb9a4bffb491d6457a0dffca (127.16.79.65:37033)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
I20260619 06:49:10.075052 1364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:46703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:49:10.104092 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:36603
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:45149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:10.441597 1559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:10.441972 1559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:10.442107 1559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:10.452389 1559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:10.452507 1559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:10.452625 1559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:49:10.462553 1559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:45149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:36603
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:10.463804 1559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:10.465132 1559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:10.478336 1565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:10.478428 1564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:10.478379 1567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:10.485235 1559 server_base.cc:1034] running on GCE node
I20260619 06:49:10.486039 1559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:10.487051 1559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:10.488281 1559 hybrid_clock.cc:648] HybridClock initialized: now 1781851750488198 us; error 53 us; skew 500 ppm
I20260619 06:49:10.488628 1559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:10 dist-test-slave-5w2x krb5kdc[1319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851750, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:10.491811 1559 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:49:10.493448 1559 webserver.cc:460] Webserver started at http://127.16.79.66:44155/ using document root <none> and password file <none>
I20260619 06:49:10.494189 1559 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:10.494334 1559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:10.494663 1559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:10.497620 1559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "b35db0430c9549e68d6be6a00af32eef"
format_stamp: "Formatted at 2026-06-19 06:49:10 on dist-test-slave-5w2x"
server_key: "54ffae995d1a2507d0cd403f45da0e56"
server_key_iv: "1d8b50bf25162a09cef504f29f8ff252"
server_key_version: "encryptionkey@0"
I20260619 06:49:10.498415 1559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "b35db0430c9549e68d6be6a00af32eef"
format_stamp: "Formatted at 2026-06-19 06:49:10 on dist-test-slave-5w2x"
server_key: "54ffae995d1a2507d0cd403f45da0e56"
server_key_iv: "1d8b50bf25162a09cef504f29f8ff252"
server_key_version: "encryptionkey@0"
I20260619 06:49:10.503576 1559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260619 06:49:10.507282 1574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:10.508776 1559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 06:49:10.508952 1559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "b35db0430c9549e68d6be6a00af32eef"
format_stamp: "Formatted at 2026-06-19 06:49:10 on dist-test-slave-5w2x"
server_key: "54ffae995d1a2507d0cd403f45da0e56"
server_key_iv: "1d8b50bf25162a09cef504f29f8ff252"
server_key_version: "encryptionkey@0"
I20260619 06:49:10.509138 1559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:10.534789 1559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:10.539184 1559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:10.539542 1559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:10.540583 1559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:10.542009 1559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:10.542088 1559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:10.542223 1559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:10.542281 1559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:10.576228 1559 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:38111
I20260619 06:49:10.576292 1686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:38111 every 8 connection(s)
I20260619 06:49:10.577615 1559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:49:10.579201 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 1559
I20260619 06:49:10.579396 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:49:10.579972 16701 external_mini_cluster.cc:1428] Setting key 7ed584b377300f2dfae76a156ff0247c
Jun 19 06:49:10 dist-test-slave-5w2x krb5kdc[1319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851750, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:49:10.601581 1689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:10.580509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:35231 (local address 127.16.79.126:36603)
0619 06:49:10.580992 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:10.580998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:10.584044 (+ 3046us) server_negotiation.cc:407] Connection header received
0619 06:49:10.584121 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:10.584129 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:10.584244 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:10.584366 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:10.586039 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:10.587266 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:10.589200 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:10.589818 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:10.589922 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:10.594651 (+ 4729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:10.594673 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:10.594676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:10.594711 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:10.597537 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:10.598672 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:10.598677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:10.598679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:10.598825 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:10.599516 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:10.599521 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:10.599523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:10.599834 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:10.599964 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:10.601048 (+ 1084us) server_negotiation.cc:299] Negotiation successful
0619 06:49:10.601303 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":238,"threads_started":1}
I20260619 06:49:10.603248 1690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:10.581122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36603 (local address 127.16.79.66:35231)
0619 06:49:10.581941 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:10.581993 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:10.583707 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:10.584607 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:10.584615 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:10.585231 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:10.585860 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:10.585879 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:10.587397 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:10.587400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:10.589036 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:10.589049 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:10.590070 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:10.590080 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:10.590462 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:10.591232 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:49:10.591277 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:10.594391 (+ 3114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:10.597738 (+ 3347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:10.597751 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:10.597774 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:10.598481 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:10.598967 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:10.598975 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:10.598981 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:10.599369 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:10.600015 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:10.600028 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:10.600569 (+ 541us) client_negotiation.cc:769] Sending connection context
0619 06:49:10.600940 (+ 371us) client_negotiation.cc:240] Negotiation successful
0619 06:49:10.601601 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":341,"threads_started":1}
I20260619 06:49:10.604933 1687 heartbeater.cc:347] Connected to a master server at 127.16.79.126:36603
I20260619 06:49:10.605320 1687 heartbeater.cc:464] Registering TS with master...
I20260619 06:49:10.606288 1687 heartbeater.cc:511] Master 127.16.79.126:36603 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
I20260619 06:49:10.608201 1364 ts_manager.cc:194] Registered new tserver with Master: b35db0430c9549e68d6be6a00af32eef (127.16.79.66:38111)
I20260619 06:49:10.609115 1364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:35231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:49:10.639813 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:36603
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:45149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:10.977029 1693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:10.977417 1693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:10.977591 1693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:10.985219 1693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:10.985375 1693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:10.985549 1693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:49:10.995857 1693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:45149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:36603
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:10.997390 1693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:10.998739 1693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:11.012085 1698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:11.012046 1701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:11.012156 1699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:11.018796 1693 server_base.cc:1034] running on GCE node
I20260619 06:49:11.019573 1693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:11.020752 1693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:11.021993 1693 hybrid_clock.cc:648] HybridClock initialized: now 1781851751021951 us; error 46 us; skew 500 ppm
I20260619 06:49:11.022332 1693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:11 dist-test-slave-5w2x krb5kdc[1319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851751, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:11.025933 1693 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:49:11.027627 1693 webserver.cc:460] Webserver started at http://127.16.79.67:39569/ using document root <none> and password file <none>
I20260619 06:49:11.028479 1693 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:11.028630 1693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:11.028954 1693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:11.031874 1693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "d18c95f5e4234983925aa08e3e0a9622"
format_stamp: "Formatted at 2026-06-19 06:49:11 on dist-test-slave-5w2x"
server_key: "c937fa728be4fcf117c469c0be07fb29"
server_key_iv: "a8676beb22f5b27076a6e26536048e38"
server_key_version: "encryptionkey@0"
I20260619 06:49:11.032728 1693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "d18c95f5e4234983925aa08e3e0a9622"
format_stamp: "Formatted at 2026-06-19 06:49:11 on dist-test-slave-5w2x"
server_key: "c937fa728be4fcf117c469c0be07fb29"
server_key_iv: "a8676beb22f5b27076a6e26536048e38"
server_key_version: "encryptionkey@0"
I20260619 06:49:11.038287 1693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:49:11.042207 1708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:11.043756 1693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:49:11.043920 1693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "d18c95f5e4234983925aa08e3e0a9622"
format_stamp: "Formatted at 2026-06-19 06:49:11 on dist-test-slave-5w2x"
server_key: "c937fa728be4fcf117c469c0be07fb29"
server_key_iv: "a8676beb22f5b27076a6e26536048e38"
server_key_version: "encryptionkey@0"
I20260619 06:49:11.044214 1693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:11.073276 1693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:11.077589 1693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:11.077894 1693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:11.078325 1553 heartbeater.cc:503] Master 127.16.79.126:36603 was elected leader, sending a full tablet report...
I20260619 06:49:11.079057 1693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:11.080761 1693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:11.080863 1693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:11.080992 1693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:11.081066 1693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:11.115101 1693 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:45541
I20260619 06:49:11.115197 1820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:45541 every 8 connection(s)
I20260619 06:49:11.116499 1693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:49:11.125759 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 1693
I20260619 06:49:11.125953 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:49:11.126498 16701 external_mini_cluster.cc:1428] Setting key e31dd058a1ced6db3dee43ea942dd103
Jun 19 06:49:11 dist-test-slave-5w2x krb5kdc[1319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851751, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:49:11.137517 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.119021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:53439 (local address 127.16.79.126:36603)
0619 06:49:11.119515 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:11.119519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:11.120349 (+ 830us) server_negotiation.cc:407] Connection header received
0619 06:49:11.122184 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:11.122189 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:11.122298 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:11.122421 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:11.123952 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.124790 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.127164 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.127753 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.127900 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.131898 (+ 3998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:11.131903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:11.131907 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:11.131942 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:11.134542 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:11.135292 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:11.135297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:11.135299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:11.135430 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:11.135842 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:11.135845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:11.135846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:11.136125 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:11.136326 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:11.137032 (+ 706us) server_negotiation.cc:299] Negotiation successful
0619 06:49:11.137234 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":277,"threads_started":1}
I20260619 06:49:11.138456 1824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.119437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:36603 (local address 127.16.79.67:53439)
0619 06:49:11.120099 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:11.120240 (+ 141us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:11.121920 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:11.122657 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:11.122666 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:11.123181 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:11.123771 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:11.123788 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.124956 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.124960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:11.126971 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:11.126987 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.128068 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.128083 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:11.128441 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.129287 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:49:11.129310 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:11.131683 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:11.134686 (+ 3003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:11.134701 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:11.134715 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:11.135161 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:11.135531 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:11.135535 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:11.135537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:11.135744 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:11.136305 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:11.136311 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:11.136647 (+ 336us) client_negotiation.cc:769] Sending connection context
0619 06:49:11.136899 (+ 252us) client_negotiation.cc:240] Negotiation successful
0619 06:49:11.137314 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":266,"threads_started":1}
I20260619 06:49:11.139954 1821 heartbeater.cc:347] Connected to a master server at 127.16.79.126:36603
I20260619 06:49:11.140364 1821 heartbeater.cc:464] Registering TS with master...
I20260619 06:49:11.141211 1821 heartbeater.cc:511] Master 127.16.79.126:36603 requested a full tablet report, sending...
I20260619 06:49:11.142913 1364 ts_manager.cc:194] Registered new tserver with Master: d18c95f5e4234983925aa08e3e0a9622 (127.16.79.67:45541)
I20260619 06:49:11.143720 1364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:53439
I20260619 06:49:11.156291 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:49:11.181069 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.163238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38944 (local address 127.16.79.126:36603)
0619 06:49:11.163452 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:11.163460 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:11.163917 (+ 457us) server_negotiation.cc:407] Connection header received
0619 06:49:11.164167 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:11.164174 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:11.164273 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:11.164393 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:11.166115 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.166980 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.169024 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.169762 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.169918 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.173454 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:11.173461 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:11.173476 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:11.173529 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:11.177069 (+ 3540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:11.177995 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:11.178001 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:11.178005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:11.178224 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:11.178696 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:11.178704 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:11.178710 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:11.179288 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:11.179541 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:11.180367 (+ 826us) server_negotiation.cc:299] Negotiation successful
0619 06:49:11.180661 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:49:11.186483 1364 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:49:11.197647 1364 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:49:11.225683 1834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.213103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:45541 (local address 127.0.0.1:38152)
0619 06:49:11.213880 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:11.213933 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:11.214117 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:11.215165 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:11.215174 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:11.215213 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:11.215664 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:11.215682 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.218063 (+ 2381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.218071 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:11.221294 (+ 3223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:11.221320 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.224283 (+ 2963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.224293 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:11.224595 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.224671 (+ 76us) client_negotiation.cc:769] Sending connection context
0619 06:49:11.224839 (+ 168us) client_negotiation.cc:240] Negotiation successful
0619 06:49:11.225141 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":284,"threads_started":1}
I20260619 06:49:11.226992 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.213467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38152 (local address 127.16.79.67:45541)
0619 06:49:11.214214 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:11.214223 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:11.214256 (+ 33us) server_negotiation.cc:407] Connection header received
0619 06:49:11.214365 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:11.214372 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:11.214634 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:11.214842 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:11.215880 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.217860 (+ 1980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.221557 (+ 3697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.223602 (+ 2045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.223770 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.224096 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:11.226289 (+ 2193us) server_negotiation.cc:299] Negotiation successful
0619 06:49:11.226549 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":553,"threads_started":1}
I20260619 06:49:11.232004 1756 tablet_service.cc:1467] Processing CreateTablet for tablet 715408fd40a543bd8c9add61f82868fc (DEFAULT_TABLE table=test-table [id=b913747b5c95469d867deb6e24d32026]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:11.233938 1756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 715408fd40a543bd8c9add61f82868fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:11.242027 1834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.214863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:38111 (local address 127.0.0.1:56612)
0619 06:49:11.225967 (+ 11104us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:11.225987 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:11.226095 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:11.233930 (+ 7835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:11.233934 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:11.233955 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:11.234120 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:11.234130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.236237 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.236243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:11.237867 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:11.237879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.241144 (+ 3265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.241151 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:11.241352 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.241390 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 06:49:11.241467 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 06:49:11.241632 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11006,"mutex_wait_us":35,"threads_started":1}
I20260619 06:49:11.242754 1838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.215458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:37033 (local address 127.0.0.1:33592)
0619 06:49:11.231016 (+ 15558us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:11.231036 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:11.231157 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:11.233713 (+ 2556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:11.233717 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:11.233743 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:11.234592 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:11.234604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.236526 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.236531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:11.239633 (+ 3102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:11.239649 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.241920 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.241926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:11.242130 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.242178 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 06:49:11.242265 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 06:49:11.242407 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15431,"thread_start_us":15260,"threads_started":1}
I20260619 06:49:11.243564 1837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.215112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56612 (local address 127.16.79.66:38111)
0619 06:49:11.233352 (+ 18240us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:11.233363 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:11.233396 (+ 33us) server_negotiation.cc:407] Connection header received
0619 06:49:11.233506 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:11.233515 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:11.233743 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:11.233939 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:11.234289 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.236016 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.238037 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.240931 (+ 2894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.242535 (+ 1604us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.242756 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:11.243047 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 06:49:11.243234 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18079,"threads_started":1}
I20260619 06:49:11.243575 1839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.216121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33592 (local address 127.16.79.65:37033)
0619 06:49:11.232754 (+ 16633us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:11.232760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:11.232792 (+ 32us) server_negotiation.cc:407] Connection header received
0619 06:49:11.232924 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:11.232932 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:11.233206 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:11.233436 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:11.234813 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.236372 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.239867 (+ 3495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.241407 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.241533 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.241772 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:11.243068 (+ 1296us) server_negotiation.cc:299] Negotiation successful
0619 06:49:11.243237 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16460,"thread_start_us":16262,"threads_started":1}
I20260619 06:49:11.247747 1488 tablet_service.cc:1467] Processing CreateTablet for tablet 715408fd40a543bd8c9add61f82868fc (DEFAULT_TABLE table=test-table [id=b913747b5c95469d867deb6e24d32026]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:11.248209 1622 tablet_service.cc:1467] Processing CreateTablet for tablet 715408fd40a543bd8c9add61f82868fc (DEFAULT_TABLE table=test-table [id=b913747b5c95469d867deb6e24d32026]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:11.249401 1488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 715408fd40a543bd8c9add61f82868fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:11.249418 1622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 715408fd40a543bd8c9add61f82868fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:11.252914 1840 tablet_bootstrap.cc:492] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622: Bootstrap starting.
I20260619 06:49:11.257331 1840 tablet_bootstrap.cc:654] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:11.258993 1840 log.cc:824] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:11.264104 1840 tablet_bootstrap.cc:492] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622: No bootstrap required, opened a new log
I20260619 06:49:11.264425 1840 ts_tablet_manager.cc:1397] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260619 06:49:11.272279 1842 tablet_bootstrap.cc:492] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef: Bootstrap starting.
I20260619 06:49:11.273468 1843 tablet_bootstrap.cc:492] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca: Bootstrap starting.
I20260619 06:49:11.276306 1840 raft_consensus.cc:348] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d18c95f5e4234983925aa08e3e0a9622" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45541 } } peers { permanent_uuid: "b35db0430c9549e68d6be6a00af32eef" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38111 } } peers { permanent_uuid: "0efa6eafbb9a4bffb491d6457a0dffca" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37033 } }
I20260619 06:49:11.276571 1842 tablet_bootstrap.cc:654] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:11.276705 1840 raft_consensus.cc:374] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:11.276803 1840 raft_consensus.cc:729] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d18c95f5e4234983925aa08e3e0a9622, State: Initialized, Role: FOLLOWER
I20260619 06:49:11.277113 1843 tablet_bootstrap.cc:654] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:11.277495 1840 consensus_queue.cc:260] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d18c95f5e4234983925aa08e3e0a9622" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45541 } } peers { permanent_uuid: "b35db0430c9549e68d6be6a00af32eef" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38111 } } peers { permanent_uuid: "0efa6eafbb9a4bffb491d6457a0dffca" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37033 } }
I20260619 06:49:11.277923 1842 log.cc:824] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:11.278476 1843 log.cc:824] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:11.278754 1821 heartbeater.cc:503] Master 127.16.79.126:36603 was elected leader, sending a full tablet report...
I20260619 06:49:11.280242 1840 ts_tablet_manager.cc:1428] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
I20260619 06:49:11.281886 1843 tablet_bootstrap.cc:492] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca: No bootstrap required, opened a new log
I20260619 06:49:11.282218 1843 ts_tablet_manager.cc:1397] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260619 06:49:11.282759 1842 tablet_bootstrap.cc:492] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef: No bootstrap required, opened a new log
I20260619 06:49:11.283061 1842 ts_tablet_manager.cc:1397] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260619 06:49:11.290612 1842 raft_consensus.cc:348] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d18c95f5e4234983925aa08e3e0a9622" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45541 } } peers { permanent_uuid: "b35db0430c9549e68d6be6a00af32eef" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38111 } } peers { permanent_uuid: "0efa6eafbb9a4bffb491d6457a0dffca" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37033 } }
I20260619 06:49:11.290613 1843 raft_consensus.cc:348] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d18c95f5e4234983925aa08e3e0a9622" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45541 } } peers { permanent_uuid: "b35db0430c9549e68d6be6a00af32eef" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38111 } } peers { permanent_uuid: "0efa6eafbb9a4bffb491d6457a0dffca" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37033 } }
I20260619 06:49:11.291004 1842 raft_consensus.cc:374] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:11.291018 1843 raft_consensus.cc:374] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:11.291103 1843 raft_consensus.cc:729] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0efa6eafbb9a4bffb491d6457a0dffca, State: Initialized, Role: FOLLOWER
I20260619 06:49:11.291103 1842 raft_consensus.cc:729] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b35db0430c9549e68d6be6a00af32eef, State: Initialized, Role: FOLLOWER
I20260619 06:49:11.291682 1843 consensus_queue.cc:260] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d18c95f5e4234983925aa08e3e0a9622" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45541 } } peers { permanent_uuid: "b35db0430c9549e68d6be6a00af32eef" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38111 } } peers { permanent_uuid: "0efa6eafbb9a4bffb491d6457a0dffca" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37033 } }
I20260619 06:49:11.291733 1842 consensus_queue.cc:260] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d18c95f5e4234983925aa08e3e0a9622" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45541 } } peers { permanent_uuid: "b35db0430c9549e68d6be6a00af32eef" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38111 } } peers { permanent_uuid: "0efa6eafbb9a4bffb491d6457a0dffca" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37033 } }
I20260619 06:49:11.293332 1842 ts_tablet_manager.cc:1428] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260619 06:49:11.293501 1843 ts_tablet_manager.cc:1428] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca: Time spent starting tablet: real 0.011s user 0.009s sys 0.003s
I20260619 06:49:11.293716 1687 heartbeater.cc:503] Master 127.16.79.126:36603 was elected leader, sending a full tablet report...
W20260619 06:49:11.293946 1554 tablet.cc:2369] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:49:11.321311 1847 raft_consensus.cc:482] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:49:11.321647 1847 raft_consensus.cc:504] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d18c95f5e4234983925aa08e3e0a9622" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45541 } } peers { permanent_uuid: "b35db0430c9549e68d6be6a00af32eef" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38111 } } peers { permanent_uuid: "0efa6eafbb9a4bffb491d6457a0dffca" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37033 } }
I20260619 06:49:11.323364 1847 leader_election.cc:288] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d18c95f5e4234983925aa08e3e0a9622 (127.16.79.67:45541), b35db0430c9549e68d6be6a00af32eef (127.16.79.66:38111)
W20260619 06:49:11.329679 1688 tablet.cc:2369] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:49:11.333179 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.323934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:60347 (local address 127.16.79.67:45541)
0619 06:49:11.324119 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:11.324126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:11.324931 (+ 805us) server_negotiation.cc:407] Connection header received
0619 06:49:11.325133 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:11.325139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:11.325223 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:11.325334 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:11.326218 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.327391 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.330617 (+ 3226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.331917 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.332086 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.332189 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:11.332781 (+ 592us) server_negotiation.cc:299] Negotiation successful
0619 06:49:11.332921 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 06:49:11.333218 1837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.323844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:37687 (local address 127.16.79.66:38111)
0619 06:49:11.324068 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:11.324076 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:11.325008 (+ 932us) server_negotiation.cc:407] Connection header received
0619 06:49:11.325132 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:11.325141 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:11.325250 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:11.325343 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:11.326241 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.327391 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.330638 (+ 3247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.331917 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.332093 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.332193 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:11.332781 (+ 588us) server_negotiation.cc:299] Negotiation successful
0619 06:49:11.332921 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 06:49:11.333421 1849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.323799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:45541 (local address 127.16.79.65:60347)
0619 06:49:11.324771 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:11.324798 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:11.324972 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:11.325410 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:11.325416 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:11.325514 (+ 98us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:11.326033 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:11.326041 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.327591 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.327597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:11.330397 (+ 2800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:11.330416 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.332219 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.332231 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:11.332530 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.332619 (+ 89us) client_negotiation.cc:769] Sending connection context
0619 06:49:11.332722 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 06:49:11.332889 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":244,"spinlock_wait_cycles":21760,"thread_start_us":509,"threads_started":1}
I20260619 06:49:11.333458 1850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.323898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:38111 (local address 127.16.79.65:37687)
0619 06:49:11.324885 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:11.324899 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:11.324993 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:11.325410 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:11.325416 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:11.325448 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:11.326006 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:11.326026 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.327591 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.327597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:11.330397 (+ 2800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:11.330416 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.332243 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.332248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:11.332542 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.332625 (+ 83us) client_negotiation.cc:769] Sending connection context
0619 06:49:11.332722 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 06:49:11.332889 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":868,"mutex_wait_us":292,"thread_start_us":489,"threads_started":1}
I20260619 06:49:11.334508 1642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "715408fd40a543bd8c9add61f82868fc" candidate_uuid: "0efa6eafbb9a4bffb491d6457a0dffca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b35db0430c9549e68d6be6a00af32eef" is_pre_election: true
I20260619 06:49:11.334656 1776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "715408fd40a543bd8c9add61f82868fc" candidate_uuid: "0efa6eafbb9a4bffb491d6457a0dffca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d18c95f5e4234983925aa08e3e0a9622" is_pre_election: true
I20260619 06:49:11.335142 1776 raft_consensus.cc:2454] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0efa6eafbb9a4bffb491d6457a0dffca in term 0.
I20260619 06:49:11.335143 1642 raft_consensus.cc:2454] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0efa6eafbb9a4bffb491d6457a0dffca in term 0.
I20260619 06:49:11.335988 1443 leader_election.cc:302] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0efa6eafbb9a4bffb491d6457a0dffca, b35db0430c9549e68d6be6a00af32eef, d18c95f5e4234983925aa08e3e0a9622; no voters:
I20260619 06:49:11.336501 1847 raft_consensus.cc:2781] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:49:11.336661 1847 raft_consensus.cc:482] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:49:11.336818 1847 raft_consensus.cc:3037] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:11.339248 1847 raft_consensus.cc:504] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d18c95f5e4234983925aa08e3e0a9622" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45541 } } peers { permanent_uuid: "b35db0430c9549e68d6be6a00af32eef" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38111 } } peers { permanent_uuid: "0efa6eafbb9a4bffb491d6457a0dffca" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37033 } }
I20260619 06:49:11.339984 1847 leader_election.cc:288] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [CANDIDATE]: Term 1 election: Requested vote from peers d18c95f5e4234983925aa08e3e0a9622 (127.16.79.67:45541), b35db0430c9549e68d6be6a00af32eef (127.16.79.66:38111)
I20260619 06:49:11.340508 1776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "715408fd40a543bd8c9add61f82868fc" candidate_uuid: "0efa6eafbb9a4bffb491d6457a0dffca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d18c95f5e4234983925aa08e3e0a9622"
I20260619 06:49:11.340576 1642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "715408fd40a543bd8c9add61f82868fc" candidate_uuid: "0efa6eafbb9a4bffb491d6457a0dffca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b35db0430c9549e68d6be6a00af32eef"
I20260619 06:49:11.340756 1776 raft_consensus.cc:3037] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:11.340848 1642 raft_consensus.cc:3037] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:11.343077 1776 raft_consensus.cc:2454] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0efa6eafbb9a4bffb491d6457a0dffca in term 1.
I20260619 06:49:11.343080 1642 raft_consensus.cc:2454] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0efa6eafbb9a4bffb491d6457a0dffca in term 1.
I20260619 06:49:11.343538 1443 leader_election.cc:302] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0efa6eafbb9a4bffb491d6457a0dffca, d18c95f5e4234983925aa08e3e0a9622; no voters:
I20260619 06:49:11.343873 1847 raft_consensus.cc:2781] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:11.344350 1847 raft_consensus.cc:686] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [term 1 LEADER]: Becoming Leader. State: Replica: 0efa6eafbb9a4bffb491d6457a0dffca, State: Running, Role: LEADER
I20260619 06:49:11.344828 1847 consensus_queue.cc:237] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d18c95f5e4234983925aa08e3e0a9622" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45541 } } peers { permanent_uuid: "b35db0430c9549e68d6be6a00af32eef" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38111 } } peers { permanent_uuid: "0efa6eafbb9a4bffb491d6457a0dffca" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37033 } }
I20260619 06:49:11.349696 1363 catalog_manager.cc:5495] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca reported cstate change: term changed from 0 to 1, leader changed from <none> to 0efa6eafbb9a4bffb491d6457a0dffca (127.16.79.65). New cstate: current_term: 1 leader_uuid: "0efa6eafbb9a4bffb491d6457a0dffca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d18c95f5e4234983925aa08e3e0a9622" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b35db0430c9549e68d6be6a00af32eef" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0efa6eafbb9a4bffb491d6457a0dffca" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37033 } health_report { overall_health: HEALTHY } } }
W20260619 06:49:11.368275 1822 tablet.cc:2369] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:49:11.383376 1839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.377829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33594 (local address 127.16.79.65:37033)
0619 06:49:11.378051 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:11.378059 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:11.378083 (+ 24us) server_negotiation.cc:407] Connection header received
0619 06:49:11.378400 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:11.378404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:11.378483 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:11.378634 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:49:11.379038 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.379762 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.381249 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.381644 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.381787 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.382284 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:49:11.382779 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:49:11.382938 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:11.383040 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 06:49:11.383121 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:49:11.391923 1776 raft_consensus.cc:1261] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622 [term 1 FOLLOWER]: Refusing update from remote peer 0efa6eafbb9a4bffb491d6457a0dffca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:49:11.391968 1642 raft_consensus.cc:1261] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef [term 1 FOLLOWER]: Refusing update from remote peer 0efa6eafbb9a4bffb491d6457a0dffca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:49:11.392915 1851 consensus_queue.cc:1035] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d18c95f5e4234983925aa08e3e0a9622" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 45541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:49:11.393231 1847 consensus_queue.cc:1035] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [LEADER]: Connected to new peer: Peer: permanent_uuid: "b35db0430c9549e68d6be6a00af32eef" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:49:11.403379 1854 mvcc.cc:205] Tried to move back new op lower bound from 7298464773690396672 to 7298464773511933952. Current Snapshot: MvccSnapshot[applied={T|T < 7298464773690396672}]
I20260619 06:49:11.419221 1364 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:49:11.419507 1364 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:49:11.423252 1364 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da442114001244e0b76a7541f9df9209: Sending DeleteTablet for 3 replicas of tablet 715408fd40a543bd8c9add61f82868fc
I20260619 06:49:11.424517 1756 tablet_service.cc:1514] Processing DeleteTablet for tablet 715408fd40a543bd8c9add61f82868fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:11 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:38152
I20260619 06:49:11.424602 1622 tablet_service.cc:1514] Processing DeleteTablet for tablet 715408fd40a543bd8c9add61f82868fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:11 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:56612
I20260619 06:49:11.425611 1488 tablet_service.cc:1514] Processing DeleteTablet for tablet 715408fd40a543bd8c9add61f82868fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:11 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:33592
I20260619 06:49:11.426115 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 1425
I20260619 06:49:11.450204 1866 tablet_replica.cc:330] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca: stopping tablet replica
I20260619 06:49:11.450946 1866 raft_consensus.cc:2229] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:49:11.451794 1866 raft_consensus.cc:2258] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:11.454289 1864 tablet_replica.cc:330] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622: stopping tablet replica
I20260619 06:49:11.454485 1866 ts_tablet_manager.cc:1905] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:11.454749 1864 raft_consensus.cc:2229] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:49:11.455277 1864 raft_consensus.cc:2258] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:11.456085 1865 tablet_replica.cc:330] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef: stopping tablet replica
I20260619 06:49:11.456529 1865 raft_consensus.cc:2229] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:49:11.457021 1865 raft_consensus.cc:2258] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:11.459363 1865 ts_tablet_manager.cc:1905] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 19 06:49:11 dist-test-slave-5w2x krb5kdc[1319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:49:11.463017 1864 ts_tablet_manager.cc:1905] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:11.463994 1866 ts_tablet_manager.cc:1918] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:11.464269 1866 log.cc:1196] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/715408fd40a543bd8c9add61f82868fc
I20260619 06:49:11.464906 1866 ts_tablet_manager.cc:1939] T 715408fd40a543bd8c9add61f82868fc P 0efa6eafbb9a4bffb491d6457a0dffca: Deleting consensus metadata
I20260619 06:49:11.466933 1349 catalog_manager.cc:4841] TS 0efa6eafbb9a4bffb491d6457a0dffca (127.16.79.65:37033): tablet 715408fd40a543bd8c9add61f82868fc (table test-table [id=b913747b5c95469d867deb6e24d32026]) successfully deleted
I20260619 06:49:11.469705 1864 ts_tablet_manager.cc:1918] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:11.469884 1864 log.cc:1196] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/715408fd40a543bd8c9add61f82868fc
I20260619 06:49:11.470438 1864 ts_tablet_manager.cc:1939] T 715408fd40a543bd8c9add61f82868fc P d18c95f5e4234983925aa08e3e0a9622: Deleting consensus metadata
I20260619 06:49:11.471148 1839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.432394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33598 (local address 127.16.79.65:37033)
0619 06:49:11.432623 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:11.432633 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:11.447839 (+ 15206us) server_negotiation.cc:407] Connection header received
0619 06:49:11.448196 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:11.448205 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:11.448323 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:11.448426 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:11.450604 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.451895 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.456063 (+ 4168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.456829 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.456948 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.462654 (+ 5706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:11.462679 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:11.462699 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:11.462757 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:11.467279 (+ 4522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:11.468284 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:11.468293 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:11.468296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:11.468480 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:11.468995 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:11.469000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:11.469003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:11.469495 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:11.469783 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:11.470437 (+ 654us) server_negotiation.cc:299] Negotiation successful
0619 06:49:11.470747 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 06:49:11.472597 1548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:11.472719 1350 catalog_manager.cc:4841] TS d18c95f5e4234983925aa08e3e0a9622 (127.16.79.67:45541): tablet 715408fd40a543bd8c9add61f82868fc (table test-table [id=b913747b5c95469d867deb6e24d32026]) successfully deleted
I20260619 06:49:11.474448 1865 ts_tablet_manager.cc:1918] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:11.474659 1865 log.cc:1196] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/715408fd40a543bd8c9add61f82868fc
I20260619 06:49:11.475237 1865 ts_tablet_manager.cc:1939] T 715408fd40a543bd8c9add61f82868fc P b35db0430c9549e68d6be6a00af32eef: Deleting consensus metadata
I20260619 06:49:11.477408 1351 catalog_manager.cc:4841] TS b35db0430c9549e68d6be6a00af32eef (127.16.79.66:38111): tablet 715408fd40a543bd8c9add61f82868fc (table test-table [id=b913747b5c95469d867deb6e24d32026]) successfully deleted
I20260619 06:49:11.581557 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 1425
I20260619 06:49:11.600106 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 1559
Jun 19 06:49:11 dist-test-slave-5w2x krb5kdc[1319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:49:11.616952 1837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.600995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56616 (local address 127.16.79.66:38111)
0619 06:49:11.601176 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:11.601181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:11.601199 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:49:11.601333 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:11.601338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:11.601399 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:11.601484 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:11.602844 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.603577 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.604908 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.605354 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.605461 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.611137 (+ 5676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:11.611156 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:11.611174 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:11.611216 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:11.614299 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:11.615024 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:11.615031 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:11.615034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:11.615167 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:11.615479 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:11.615482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:11.615484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:11.615892 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:11.616378 (+ 486us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:11.616502 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 06:49:11.616680 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:49:11.617712 1682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:11.708700 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 1559
I20260619 06:49:11.725512 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 1693
Jun 19 06:49:11 dist-test-slave-5w2x krb5kdc[1319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:49:11.741080 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:11.726323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38160 (local address 127.16.79.67:45541)
0619 06:49:11.726490 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:11.726494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:11.726514 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:49:11.726636 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:11.726641 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:11.726718 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:11.726802 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:11.728045 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.728829 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.730156 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:11.730548 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:11.730645 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:11.735263 (+ 4618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:11.735280 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:11.735296 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:11.735333 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:11.738250 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:11.739002 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:11.739010 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:11.739012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:11.739139 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:11.739550 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:11.739554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:11.739556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:11.739969 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:11.740322 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:11.740605 (+ 283us) server_negotiation.cc:299] Negotiation successful
0619 06:49:11.740818 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 06:49:11.741844 1816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:11.833174 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 1693
I20260619 06:49:11.849817 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 1332
I20260619 06:49:11.851003 1394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:11.964232 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 1332
2026-06-19T06:49:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4336 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T06:49:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:12Z Disabled control of system clock
I20260619 06:49:12.043457 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:43557
--webserver_interface=127.16.79.126
--webserver_port=0
--builtin_ntp_servers=127.16.79.84:46485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:43557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 06:49:12.390844 1875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:12.391206 1875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:12.391392 1875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:12.398890 1875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:12.399010 1875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:12.399096 1875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:12.399199 1875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:12.399271 1875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:12.408846 1875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:43557
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:43557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:12.410230 1875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:12.412111 1875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:12.424453 1880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:12.424537 1881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:12.424497 1883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:12.425616 1875 server_base.cc:1034] running on GCE node
I20260619 06:49:12.426578 1875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:12.428228 1875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:12.429464 1875 hybrid_clock.cc:648] HybridClock initialized: now 1781851752429423 us; error 43 us; skew 500 ppm
I20260619 06:49:12.429854 1875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:49:12.435668 1875 webserver.cc:460] Webserver started at http://127.16.79.126:44739/ using document root <none> and password file <none>
I20260619 06:49:12.436555 1875 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:12.436712 1875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:12.437050 1875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:12.440254 1875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "e6650ccc8034488bab3ec4e936f085cf"
format_stamp: "Formatted at 2026-06-19 06:49:12 on dist-test-slave-5w2x"
server_key: "3e11f7e7bf9acb9cc0fb4e87c9ee269e"
server_key_iv: "1414de596702fe25ba7b4c257b75cf87"
server_key_version: "encryptionkey@0"
I20260619 06:49:12.441152 1875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "e6650ccc8034488bab3ec4e936f085cf"
format_stamp: "Formatted at 2026-06-19 06:49:12 on dist-test-slave-5w2x"
server_key: "3e11f7e7bf9acb9cc0fb4e87c9ee269e"
server_key_iv: "1414de596702fe25ba7b4c257b75cf87"
server_key_version: "encryptionkey@0"
I20260619 06:49:12.446686 1875 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260619 06:49:12.450521 1889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:12.452282 1875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:49:12.452481 1875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "e6650ccc8034488bab3ec4e936f085cf"
format_stamp: "Formatted at 2026-06-19 06:49:12 on dist-test-slave-5w2x"
server_key: "3e11f7e7bf9acb9cc0fb4e87c9ee269e"
server_key_iv: "1414de596702fe25ba7b4c257b75cf87"
server_key_version: "encryptionkey@0"
I20260619 06:49:12.452708 1875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:12.466672 1875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:12.467546 1875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:12.467857 1875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:12.487520 1875 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:43557
I20260619 06:49:12.487527 1940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:43557 every 8 connection(s)
I20260619 06:49:12.489130 1875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:12.492995 1941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:12.499078 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 1875
I20260619 06:49:12.499258 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:12.499737 16701 external_mini_cluster.cc:1428] Setting key 143bddcd95b0e1b6ead164ade3c40cb4
I20260619 06:49:12.504011 1941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf: Bootstrap starting.
I20260619 06:49:12.507583 1941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:12.508726 1941 log.cc:824] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:12.511751 1941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf: No bootstrap required, opened a new log
I20260619 06:49:12.513731 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:12.501891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54190 (local address 127.16.79.126:43557)
0619 06:49:12.502906 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:12.502923 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:12.502963 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:49:12.504781 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:12.504804 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:12.505288 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:12.505630 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:12.506295 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:12.507592 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:12.509179 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:12.509880 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:12.510079 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:12.511316 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:12.511324 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:12.511338 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:12.511952 (+ 614us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:12.512064 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:12.512087 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:12.512280 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:12.512494 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 06:49:12.512808 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":386,"threads_started":1}
I20260619 06:49:12.520624 1941 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6650ccc8034488bab3ec4e936f085cf" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43557 } }
I20260619 06:49:12.521099 1941 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:12.521250 1941 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6650ccc8034488bab3ec4e936f085cf, State: Initialized, Role: FOLLOWER
I20260619 06:49:12.521863 1941 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6650ccc8034488bab3ec4e936f085cf" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43557 } }
I20260619 06:49:12.522094 1941 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:12.522230 1941 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:12.522436 1941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:12.524772 1941 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6650ccc8034488bab3ec4e936f085cf" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43557 } }
I20260619 06:49:12.525365 1941 leader_election.cc:302] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6650ccc8034488bab3ec4e936f085cf; no voters:
I20260619 06:49:12.525962 1941 leader_election.cc:288] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:12.526147 1946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:12.527007 1946 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [term 1 LEADER]: Becoming Leader. State: Replica: e6650ccc8034488bab3ec4e936f085cf, State: Running, Role: LEADER
I20260619 06:49:12.527510 1946 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6650ccc8034488bab3ec4e936f085cf" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43557 } }
I20260619 06:49:12.528266 1941 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:12.530591 1947 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6650ccc8034488bab3ec4e936f085cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6650ccc8034488bab3ec4e936f085cf" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43557 } } }
I20260619 06:49:12.530622 1948 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6650ccc8034488bab3ec4e936f085cf. Latest consensus state: current_term: 1 leader_uuid: "e6650ccc8034488bab3ec4e936f085cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6650ccc8034488bab3ec4e936f085cf" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43557 } } }
I20260619 06:49:12.531032 1947 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:12.531068 1948 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:12.535121 1953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:12.539388 1953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:12.549014 1953 catalog_manager.cc:1347] Generated new cluster ID: 378a4aa2894c44c9bcc4bde085ac76b4
I20260619 06:49:12.549150 1953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:12.568625 1953 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:12.570564 1953 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:12.583673 1953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf: Generated new TSK 0
I20260619 06:49:12.584836 1953 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:12.597237 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43557
--builtin_ntp_servers=127.16.79.84:46485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 06:49:12.937135 1965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:12.937433 1965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:12.937580 1965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:12.945176 1965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:12.945297 1965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:12.945483 1965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:49:12.955094 1965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:12.956566 1965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:12.958436 1965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:12.972589 1970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:12.972612 1971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:12.972568 1973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:12.973435 1965 server_base.cc:1034] running on GCE node
I20260619 06:49:12.974057 1965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:12.975057 1965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:12.976259 1965 hybrid_clock.cc:648] HybridClock initialized: now 1781851752976177 us; error 74 us; skew 500 ppm
I20260619 06:49:12.976648 1965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:49:12.979212 1965 webserver.cc:460] Webserver started at http://127.16.79.65:35081/ using document root <none> and password file <none>
I20260619 06:49:12.980000 1965 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:12.980089 1965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:12.980510 1965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:12.983529 1965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "f447c357a6294e22a40921d7f03a83e0"
format_stamp: "Formatted at 2026-06-19 06:49:12 on dist-test-slave-5w2x"
server_key: "284c5575e247920bf8a60389fa5df22f"
server_key_iv: "eac473fd416a1897913caccc54140dc0"
server_key_version: "encryptionkey@0"
I20260619 06:49:12.984428 1965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "f447c357a6294e22a40921d7f03a83e0"
format_stamp: "Formatted at 2026-06-19 06:49:12 on dist-test-slave-5w2x"
server_key: "284c5575e247920bf8a60389fa5df22f"
server_key_iv: "eac473fd416a1897913caccc54140dc0"
server_key_version: "encryptionkey@0"
I20260619 06:49:12.990166 1965 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260619 06:49:12.994390 1979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:12.995872 1965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:49:12.996076 1965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "f447c357a6294e22a40921d7f03a83e0"
format_stamp: "Formatted at 2026-06-19 06:49:12 on dist-test-slave-5w2x"
server_key: "284c5575e247920bf8a60389fa5df22f"
server_key_iv: "eac473fd416a1897913caccc54140dc0"
server_key_version: "encryptionkey@0"
I20260619 06:49:12.996385 1965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:13.021386 1965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:13.022293 1965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:13.022624 1965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:13.023643 1965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:13.025290 1965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:13.025420 1965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:13.025555 1965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:13.025669 1965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:13.060077 1965 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:35377
I20260619 06:49:13.060205 2091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:35377 every 8 connection(s)
I20260619 06:49:13.061480 1965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:49:13.062398 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 1965
I20260619 06:49:13.062578 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:49:13.063159 16701 external_mini_cluster.cc:1428] Setting key 02667f5fc86db821d28c29a3d077d805
I20260619 06:49:13.066651 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43557
--builtin_ntp_servers=127.16.79.84:46485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 06:49:13.075255 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:13.064632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:53865 (local address 127.16.79.126:43557)
0619 06:49:13.065131 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:13.065137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:13.066572 (+ 1435us) server_negotiation.cc:407] Connection header received
0619 06:49:13.068515 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:13.068521 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:13.068608 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:13.068743 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:13.069864 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:13.070617 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:13.072297 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:13.072731 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:13.072875 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:13.074381 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:13.074387 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:13.074392 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:13.074423 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:13.074474 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:13.074486 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:13.074582 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:13.074811 (+ 229us) server_negotiation.cc:299] Negotiation successful
0619 06:49:13.074916 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":249,"threads_started":1}
I20260619 06:49:13.076411 2095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:13.065397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:43557 (local address 127.16.79.65:53865)
0619 06:49:13.066427 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:13.066464 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:13.068245 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:13.068991 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:13.069000 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:13.069102 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:13.069695 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:13.069712 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:13.070793 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:13.070797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:13.072055 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:13.072068 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:13.072966 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:13.072975 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:13.073234 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:13.073999 (+ 765us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:49:13.074002 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:13.074044 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:49:13.074182 (+ 138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:13.074581 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:13.074594 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:13.074605 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 06:49:13.074803 (+ 198us) client_negotiation.cc:240] Negotiation successful
0619 06:49:13.075236 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":358,"threads_started":1}
I20260619 06:49:13.078035 2092 heartbeater.cc:347] Connected to a master server at 127.16.79.126:43557
I20260619 06:49:13.078364 2092 heartbeater.cc:464] Registering TS with master...
I20260619 06:49:13.079149 2092 heartbeater.cc:511] Master 127.16.79.126:43557 requested a full tablet report, sending...
I20260619 06:49:13.081439 1906 ts_manager.cc:194] Registered new tserver with Master: f447c357a6294e22a40921d7f03a83e0 (127.16.79.65:35377)
I20260619 06:49:13.083204 1906 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.79.65:53865
W20260619 06:49:13.402684 2096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:13.403010 2096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:13.403168 2096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:13.410316 2096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:13.410458 2096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:13.410643 2096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:49:13.419725 2096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:13.421083 2096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:13.422842 2096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:13.436008 2104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:13.436398 2101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:13.435987 2102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:13.436869 2096 server_base.cc:1034] running on GCE node
I20260619 06:49:13.437529 2096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:13.438548 2096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:13.439811 2096 hybrid_clock.cc:648] HybridClock initialized: now 1781851753439755 us; error 42 us; skew 500 ppm
I20260619 06:49:13.440197 2096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:49:13.442502 2096 webserver.cc:460] Webserver started at http://127.16.79.66:45277/ using document root <none> and password file <none>
I20260619 06:49:13.443200 2096 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:13.443331 2096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:13.443655 2096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:13.446534 2096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "92f899461d3842f8bea6096a5f51a089"
format_stamp: "Formatted at 2026-06-19 06:49:13 on dist-test-slave-5w2x"
server_key: "006f11efcecc6f3fff468e49f4492560"
server_key_iv: "b0cc2c4849c2fc392e752e3936453db9"
server_key_version: "encryptionkey@0"
I20260619 06:49:13.447319 2096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "92f899461d3842f8bea6096a5f51a089"
format_stamp: "Formatted at 2026-06-19 06:49:13 on dist-test-slave-5w2x"
server_key: "006f11efcecc6f3fff468e49f4492560"
server_key_iv: "b0cc2c4849c2fc392e752e3936453db9"
server_key_version: "encryptionkey@0"
I20260619 06:49:13.452482 2096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260619 06:49:13.455909 2110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:13.457465 2096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:49:13.457649 2096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "92f899461d3842f8bea6096a5f51a089"
format_stamp: "Formatted at 2026-06-19 06:49:13 on dist-test-slave-5w2x"
server_key: "006f11efcecc6f3fff468e49f4492560"
server_key_iv: "b0cc2c4849c2fc392e752e3936453db9"
server_key_version: "encryptionkey@0"
I20260619 06:49:13.457876 2096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:13.474893 2096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:13.475893 2096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:13.476326 2096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:13.477463 2096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:13.479168 2096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:13.479267 2096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:13.479437 2096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:13.479518 2096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:13.517838 2096 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:36955
I20260619 06:49:13.517858 2222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:36955 every 8 connection(s)
I20260619 06:49:13.519735 2096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:49:13.522349 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 2096
I20260619 06:49:13.522557 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:49:13.523116 16701 external_mini_cluster.cc:1428] Setting key 2a453bc5e4e64515d56ca463de630f4a
I20260619 06:49:13.525946 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43557
--builtin_ntp_servers=127.16.79.84:46485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 06:49:13.534686 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:13.522752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:49581 (local address 127.16.79.126:43557)
0619 06:49:13.522985 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:13.522993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:13.524890 (+ 1897us) server_negotiation.cc:407] Connection header received
0619 06:49:13.527107 (+ 2217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:13.527114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:13.527216 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:13.527342 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:13.528752 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:13.529591 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:13.531336 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:13.531809 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:13.531960 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:13.533834 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:13.533839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:13.533843 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:13.533865 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:13.533901 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:13.533914 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:13.533996 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:13.534328 (+ 332us) server_negotiation.cc:299] Negotiation successful
0619 06:49:13.534416 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 06:49:13.536369 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:13.523431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:43557 (local address 127.16.79.66:49581)
0619 06:49:13.524721 (+ 1290us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:13.524773 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:13.526760 (+ 1987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:13.527629 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:13.527638 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:13.527758 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:13.528558 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:13.528580 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:13.529759 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:13.529765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:13.531160 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:13.531174 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:13.532062 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:13.532070 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:13.532465 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:13.533400 (+ 935us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:49:13.533407 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:13.533483 (+ 76us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:49:13.533629 (+ 146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:13.534009 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:13.534023 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:13.534035 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 06:49:13.534288 (+ 253us) client_negotiation.cc:240] Negotiation successful
0619 06:49:13.534775 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":402,"threads_started":1}
I20260619 06:49:13.538196 2223 heartbeater.cc:347] Connected to a master server at 127.16.79.126:43557
I20260619 06:49:13.538539 2223 heartbeater.cc:464] Registering TS with master...
I20260619 06:49:13.539348 2223 heartbeater.cc:511] Master 127.16.79.126:43557 requested a full tablet report, sending...
I20260619 06:49:13.540822 1906 ts_manager.cc:194] Registered new tserver with Master: 92f899461d3842f8bea6096a5f51a089 (127.16.79.66:36955)
I20260619 06:49:13.541639 1906 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.79.66:49581
W20260619 06:49:13.860740 2226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:13.861047 2226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:13.861222 2226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:13.869171 2226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:13.869279 2226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:13.869418 2226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:49:13.878546 2226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:46485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:43557
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:13.879892 2226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:13.881822 2226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:13.894925 2231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:13.894894 2234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:13.894862 2232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:13.895648 2226 server_base.cc:1034] running on GCE node
I20260619 06:49:13.896396 2226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:13.897379 2226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:13.898603 2226 hybrid_clock.cc:648] HybridClock initialized: now 1781851753898551 us; error 44 us; skew 500 ppm
I20260619 06:49:13.898936 2226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:49:13.901602 2226 webserver.cc:460] Webserver started at http://127.16.79.67:36301/ using document root <none> and password file <none>
I20260619 06:49:13.902415 2226 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:13.902546 2226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:13.902897 2226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:13.905944 2226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "8abd1ea6dd3a4a0fa3d2e663327e23c5"
format_stamp: "Formatted at 2026-06-19 06:49:13 on dist-test-slave-5w2x"
server_key: "87f343af5f2791cc14c0579e91f55685"
server_key_iv: "49131bb1c922b0a9e1065ad662168898"
server_key_version: "encryptionkey@0"
I20260619 06:49:13.906788 2226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "8abd1ea6dd3a4a0fa3d2e663327e23c5"
format_stamp: "Formatted at 2026-06-19 06:49:13 on dist-test-slave-5w2x"
server_key: "87f343af5f2791cc14c0579e91f55685"
server_key_iv: "49131bb1c922b0a9e1065ad662168898"
server_key_version: "encryptionkey@0"
I20260619 06:49:13.912314 2226 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:49:13.916096 2240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:13.917574 2226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:49:13.917762 2226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "8abd1ea6dd3a4a0fa3d2e663327e23c5"
format_stamp: "Formatted at 2026-06-19 06:49:13 on dist-test-slave-5w2x"
server_key: "87f343af5f2791cc14c0579e91f55685"
server_key_iv: "49131bb1c922b0a9e1065ad662168898"
server_key_version: "encryptionkey@0"
I20260619 06:49:13.917994 2226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:13.940724 2226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:13.941620 2226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:13.941953 2226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:13.943069 2226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:13.944711 2226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:13.944837 2226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:13.944962 2226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:13.945055 2226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:13.980000 2226 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:34731
I20260619 06:49:13.980094 2352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:34731 every 8 connection(s)
I20260619 06:49:13.981397 2226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:49:13.983239 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 2226
I20260619 06:49:13.983423 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:49:13.983925 16701 external_mini_cluster.cc:1428] Setting key add96985750dbbe63eea7db4bbdf7caf
I20260619 06:49:13.994375 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:13.984199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:58715 (local address 127.16.79.126:43557)
0619 06:49:13.984422 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:13.984429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:13.986142 (+ 1713us) server_negotiation.cc:407] Connection header received
0619 06:49:13.987960 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:13.987964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:13.988034 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:13.988115 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:13.989307 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:13.990058 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:13.991636 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:13.992035 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:13.992207 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:13.993562 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:13.993567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:13.993570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:13.993591 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:13.993641 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:13.993654 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:13.993741 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:13.993995 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 06:49:13.994089 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:49:13.995380 2355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:13.984862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:43557 (local address 127.16.79.67:58715)
0619 06:49:13.985972 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:13.986025 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:13.987706 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:13.988429 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:13.988438 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:13.988533 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:13.989128 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:13.989144 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:13.990230 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:13.990234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:13.991491 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:13.991503 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:13.992309 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:13.992314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:13.992527 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:13.993214 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:49:13.993217 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:13.993259 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:49:13.993347 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:13.993756 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:13.993764 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:13.993769 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 06:49:13.993910 (+ 141us) client_negotiation.cc:240] Negotiation successful
0619 06:49:13.994259 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":379,"threads_started":1}
I20260619 06:49:13.996999 2353 heartbeater.cc:347] Connected to a master server at 127.16.79.126:43557
I20260619 06:49:13.997328 2353 heartbeater.cc:464] Registering TS with master...
I20260619 06:49:13.998193 2353 heartbeater.cc:511] Master 127.16.79.126:43557 requested a full tablet report, sending...
I20260619 06:49:13.999684 1906 ts_manager.cc:194] Registered new tserver with Master: 8abd1ea6dd3a4a0fa3d2e663327e23c5 (127.16.79.67:34731)
I20260619 06:49:14.000588 1906 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.79.67:58715
I20260619 06:49:14.011982 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:49:14.022605 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:14.016406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54198 (local address 127.16.79.126:43557)
0619 06:49:14.016673 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:14.016680 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:14.016959 (+ 279us) server_negotiation.cc:407] Connection header received
0619 06:49:14.017101 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:14.017105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:14.017219 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:14.017307 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:14.017823 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.018573 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.020051 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.020547 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.020650 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:14.021745 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:14.021752 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:14.021757 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:14.021781 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:14.021822 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:14.021847 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:14.021898 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:14.022131 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 06:49:14.022220 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260619 06:49:14.026533 1906 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:49:14.034096 1906 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:49:14.074039 2365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:14.047022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:34731 (local address 127.0.0.1:52656)
0619 06:49:14.062104 (+ 15082us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:14.062160 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:14.062416 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:14.063603 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:14.063611 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:14.063646 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:14.064112 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:14.064129 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.066253 (+ 2124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.066259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:14.069268 (+ 3009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:14.069293 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.072491 (+ 3198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.072500 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:14.072834 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:14.072909 (+ 75us) client_negotiation.cc:769] Sending connection context
0619 06:49:14.073071 (+ 162us) client_negotiation.cc:240] Negotiation successful
0619 06:49:14.073355 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14645,"mutex_wait_us":4,"spinlock_wait_cycles":10112,"thread_start_us":21678,"threads_started":1}
I20260619 06:49:14.075405 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:14.047273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52656 (local address 127.16.79.67:34731)
0619 06:49:14.060805 (+ 13532us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:14.060818 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:14.062624 (+ 1806us) server_negotiation.cc:407] Connection header received
0619 06:49:14.062750 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:14.062758 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:14.063053 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:14.063254 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:14.064377 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.066072 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.069605 (+ 3533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.071687 (+ 2082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.071879 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:14.072317 (+ 438us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:14.074598 (+ 2281us) server_negotiation.cc:299] Negotiation successful
0619 06:49:14.074898 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13260,"thread_start_us":14150,"threads_started":1}
I20260619 06:49:14.076182 2368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:14.047757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52456 (local address 127.16.79.65:35377)
0619 06:49:14.061392 (+ 13635us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:14.061403 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:14.062622 (+ 1219us) server_negotiation.cc:407] Connection header received
0619 06:49:14.062771 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:14.062782 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:14.063133 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:14.063382 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:14.064661 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.066903 (+ 2242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.070574 (+ 3671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.072705 (+ 2131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.072887 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:14.073279 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:14.075578 (+ 2299us) server_negotiation.cc:299] Negotiation successful
0619 06:49:14.075777 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13420,"thread_start_us":13244,"threads_started":1}
I20260619 06:49:14.074725 2369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:14.047165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:35377 (local address 127.0.0.1:52456)
0619 06:49:14.062128 (+ 14963us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:14.062160 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:14.062416 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:14.063703 (+ 1287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:14.063707 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:14.063739 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:14.064370 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:14.064390 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.067134 (+ 2744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.067142 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:14.070245 (+ 3103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:14.070269 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.073468 (+ 3199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.073478 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:14.073785 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:14.073848 (+ 63us) client_negotiation.cc:769] Sending connection context
0619 06:49:14.073952 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 06:49:14.074182 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14827,"spinlock_wait_cycles":15232,"threads_started":1}
I20260619 06:49:14.080194 2288 tablet_service.cc:1467] Processing CreateTablet for tablet dcc65cd4f9144960b2683ba5bdb33e51 (DEFAULT_TABLE table=test-table [id=608ac552c9e944eca4872c57570e7606]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:14.080211 2027 tablet_service.cc:1467] Processing CreateTablet for tablet dcc65cd4f9144960b2683ba5bdb33e51 (DEFAULT_TABLE table=test-table [id=608ac552c9e944eca4872c57570e7606]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:14.081622 2288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcc65cd4f9144960b2683ba5bdb33e51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:14.081684 2027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcc65cd4f9144960b2683ba5bdb33e51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:14.086419 2366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:14.050507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:36955 (local address 127.0.0.1:34422)
0619 06:49:14.075233 (+ 24726us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:14.075255 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:14.075383 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:14.077271 (+ 1888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:14.077276 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:14.077307 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:14.077692 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:14.077704 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.080688 (+ 2984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.080693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:14.083423 (+ 2730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:14.083443 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.085558 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.085566 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:14.085762 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:14.085801 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 06:49:14.085912 (+ 111us) client_negotiation.cc:240] Negotiation successful
0619 06:49:14.086068 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24587,"thread_start_us":11320,"threads_started":1}
I20260619 06:49:14.086525 2370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:14.051356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34422 (local address 127.16.79.66:36955)
0619 06:49:14.076584 (+ 25228us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:14.076593 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:14.076625 (+ 32us) server_negotiation.cc:407] Connection header received
0619 06:49:14.076732 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:14.076739 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:14.077030 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:14.077257 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:14.078831 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.080523 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.083705 (+ 3182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.085326 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.085484 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:14.085746 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:14.085977 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 06:49:14.086165 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25029,"thread_start_us":26008,"threads_started":1}
I20260619 06:49:14.086689 2092 heartbeater.cc:503] Master 127.16.79.126:43557 was elected leader, sending a full tablet report...
I20260619 06:49:14.091377 2158 tablet_service.cc:1467] Processing CreateTablet for tablet dcc65cd4f9144960b2683ba5bdb33e51 (DEFAULT_TABLE table=test-table [id=608ac552c9e944eca4872c57570e7606]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:14.093091 2158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcc65cd4f9144960b2683ba5bdb33e51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:14.095870 2371 tablet_bootstrap.cc:492] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0: Bootstrap starting.
I20260619 06:49:14.098117 2372 tablet_bootstrap.cc:492] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5: Bootstrap starting.
I20260619 06:49:14.101080 2371 tablet_bootstrap.cc:654] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:14.101959 2372 tablet_bootstrap.cc:654] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:14.102532 2371 log.cc:824] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:14.103024 2372 log.cc:824] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:14.107519 2372 tablet_bootstrap.cc:492] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5: No bootstrap required, opened a new log
I20260619 06:49:14.107523 2371 tablet_bootstrap.cc:492] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0: No bootstrap required, opened a new log
I20260619 06:49:14.107846 2372 ts_tablet_manager.cc:1397] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260619 06:49:14.107846 2371 ts_tablet_manager.cc:1397] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260619 06:49:14.109771 2375 tablet_bootstrap.cc:492] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089: Bootstrap starting.
I20260619 06:49:14.113178 2375 tablet_bootstrap.cc:654] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:14.114539 2375 log.cc:824] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:14.119035 2375 tablet_bootstrap.cc:492] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089: No bootstrap required, opened a new log
I20260619 06:49:14.119351 2375 ts_tablet_manager.cc:1397] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260619 06:49:14.119087 2372 raft_consensus.cc:348] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abd1ea6dd3a4a0fa3d2e663327e23c5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 34731 } } peers { permanent_uuid: "f447c357a6294e22a40921d7f03a83e0" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35377 } } peers { permanent_uuid: "92f899461d3842f8bea6096a5f51a089" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36955 } }
I20260619 06:49:14.119091 2371 raft_consensus.cc:348] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abd1ea6dd3a4a0fa3d2e663327e23c5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 34731 } } peers { permanent_uuid: "f447c357a6294e22a40921d7f03a83e0" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35377 } } peers { permanent_uuid: "92f899461d3842f8bea6096a5f51a089" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36955 } }
I20260619 06:49:14.119570 2371 raft_consensus.cc:374] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:14.119612 2372 raft_consensus.cc:374] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:14.119705 2372 raft_consensus.cc:729] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8abd1ea6dd3a4a0fa3d2e663327e23c5, State: Initialized, Role: FOLLOWER
I20260619 06:49:14.119738 2371 raft_consensus.cc:729] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f447c357a6294e22a40921d7f03a83e0, State: Initialized, Role: FOLLOWER
I20260619 06:49:14.120424 2371 consensus_queue.cc:260] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abd1ea6dd3a4a0fa3d2e663327e23c5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 34731 } } peers { permanent_uuid: "f447c357a6294e22a40921d7f03a83e0" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35377 } } peers { permanent_uuid: "92f899461d3842f8bea6096a5f51a089" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36955 } }
I20260619 06:49:14.120420 2372 consensus_queue.cc:260] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abd1ea6dd3a4a0fa3d2e663327e23c5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 34731 } } peers { permanent_uuid: "f447c357a6294e22a40921d7f03a83e0" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35377 } } peers { permanent_uuid: "92f899461d3842f8bea6096a5f51a089" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36955 } }
I20260619 06:49:14.122066 2353 heartbeater.cc:503] Master 127.16.79.126:43557 was elected leader, sending a full tablet report...
I20260619 06:49:14.122357 2371 ts_tablet_manager.cc:1428] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
I20260619 06:49:14.122612 2372 ts_tablet_manager.cc:1428] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260619 06:49:14.129647 2375 raft_consensus.cc:348] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abd1ea6dd3a4a0fa3d2e663327e23c5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 34731 } } peers { permanent_uuid: "f447c357a6294e22a40921d7f03a83e0" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35377 } } peers { permanent_uuid: "92f899461d3842f8bea6096a5f51a089" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36955 } }
I20260619 06:49:14.130091 2375 raft_consensus.cc:374] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:14.130193 2375 raft_consensus.cc:729] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f899461d3842f8bea6096a5f51a089, State: Initialized, Role: FOLLOWER
I20260619 06:49:14.130846 2375 consensus_queue.cc:260] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abd1ea6dd3a4a0fa3d2e663327e23c5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 34731 } } peers { permanent_uuid: "f447c357a6294e22a40921d7f03a83e0" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35377 } } peers { permanent_uuid: "92f899461d3842f8bea6096a5f51a089" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36955 } }
I20260619 06:49:14.131944 2223 heartbeater.cc:503] Master 127.16.79.126:43557 was elected leader, sending a full tablet report...
I20260619 06:49:14.132695 2375 ts_tablet_manager.cc:1428] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089: Time spent starting tablet: real 0.013s user 0.009s sys 0.006s
W20260619 06:49:14.233204 2354 tablet.cc:2369] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:49:14.272351 2224 tablet.cc:2369] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:49:14.314267 2093 tablet.cc:2369] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:49:14.368939 2379 raft_consensus.cc:482] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:49:14.369295 2379 raft_consensus.cc:504] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abd1ea6dd3a4a0fa3d2e663327e23c5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 34731 } } peers { permanent_uuid: "f447c357a6294e22a40921d7f03a83e0" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35377 } } peers { permanent_uuid: "92f899461d3842f8bea6096a5f51a089" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36955 } }
I20260619 06:49:14.371115 2379 leader_election.cc:288] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8abd1ea6dd3a4a0fa3d2e663327e23c5 (127.16.79.67:34731), f447c357a6294e22a40921d7f03a83e0 (127.16.79.65:35377)
I20260619 06:49:14.378410 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:14.371847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:43521 (local address 127.16.79.67:34731)
0619 06:49:14.372046 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:14.372054 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:14.372935 (+ 881us) server_negotiation.cc:407] Connection header received
0619 06:49:14.373156 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:14.373162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:14.373258 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:14.373368 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:14.373999 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.374800 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.376682 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.377456 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.377609 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:14.377700 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:14.378021 (+ 321us) server_negotiation.cc:299] Negotiation successful
0619 06:49:14.378153 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 06:49:14.378427 2368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:14.371647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:48711 (local address 127.16.79.65:35377)
0619 06:49:14.372083 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:14.372093 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:14.372677 (+ 584us) server_negotiation.cc:407] Connection header received
0619 06:49:14.372873 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:14.372881 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:14.372978 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:14.373073 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:14.373741 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.374536 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.376728 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.377475 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.377677 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:14.377747 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:14.378071 (+ 324us) server_negotiation.cc:299] Negotiation successful
0619 06:49:14.378198 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260619 06:49:14.378531 2380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:14.371582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:35377 (local address 127.16.79.66:48711)
0619 06:49:14.372542 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:14.372567 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:14.372726 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:14.373130 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:14.373135 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:14.373163 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:14.373587 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:14.373599 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.374669 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.374675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:14.376490 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:14.376502 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.377696 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.377702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:14.377894 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:14.377917 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 06:49:14.378025 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 06:49:14.378153 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":109,"thread_start_us":819,"threads_started":1}
I20260619 06:49:14.378543 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:14.371610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:34731 (local address 127.16.79.66:43521)
0619 06:49:14.372810 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:14.372828 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:14.372958 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:14.373398 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:14.373401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:14.373421 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:14.373821 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:14.373835 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.374940 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.374944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:14.376486 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:14.376498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.377660 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.377670 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:14.377868 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:14.377892 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 06:49:14.377972 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 06:49:14.378134 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1037,"mutex_wait_us":22,"thread_start_us":473,"threads_started":1}
I20260619 06:49:14.379689 2047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcc65cd4f9144960b2683ba5bdb33e51" candidate_uuid: "92f899461d3842f8bea6096a5f51a089" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f447c357a6294e22a40921d7f03a83e0" is_pre_election: true
I20260619 06:49:14.380120 2308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcc65cd4f9144960b2683ba5bdb33e51" candidate_uuid: "92f899461d3842f8bea6096a5f51a089" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8abd1ea6dd3a4a0fa3d2e663327e23c5" is_pre_election: true
I20260619 06:49:14.380393 2047 raft_consensus.cc:2454] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92f899461d3842f8bea6096a5f51a089 in term 0.
I20260619 06:49:14.380671 2308 raft_consensus.cc:2454] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92f899461d3842f8bea6096a5f51a089 in term 0.
I20260619 06:49:14.381137 2114 leader_election.cc:302] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92f899461d3842f8bea6096a5f51a089, f447c357a6294e22a40921d7f03a83e0; no voters:
I20260619 06:49:14.381558 2379 raft_consensus.cc:2781] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:49:14.381740 2379 raft_consensus.cc:482] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:49:14.381853 2379 raft_consensus.cc:3037] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:14.384045 2379 raft_consensus.cc:504] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abd1ea6dd3a4a0fa3d2e663327e23c5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 34731 } } peers { permanent_uuid: "f447c357a6294e22a40921d7f03a83e0" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35377 } } peers { permanent_uuid: "92f899461d3842f8bea6096a5f51a089" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36955 } }
I20260619 06:49:14.384753 2379 leader_election.cc:288] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [CANDIDATE]: Term 1 election: Requested vote from peers 8abd1ea6dd3a4a0fa3d2e663327e23c5 (127.16.79.67:34731), f447c357a6294e22a40921d7f03a83e0 (127.16.79.65:35377)
I20260619 06:49:14.385393 2047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcc65cd4f9144960b2683ba5bdb33e51" candidate_uuid: "92f899461d3842f8bea6096a5f51a089" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f447c357a6294e22a40921d7f03a83e0"
I20260619 06:49:14.385399 2308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcc65cd4f9144960b2683ba5bdb33e51" candidate_uuid: "92f899461d3842f8bea6096a5f51a089" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8abd1ea6dd3a4a0fa3d2e663327e23c5"
I20260619 06:49:14.385624 2308 raft_consensus.cc:3037] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:14.385624 2047 raft_consensus.cc:3037] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:14.388945 2308 raft_consensus.cc:2454] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92f899461d3842f8bea6096a5f51a089 in term 1.
I20260619 06:49:14.388942 2047 raft_consensus.cc:2454] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92f899461d3842f8bea6096a5f51a089 in term 1.
I20260619 06:49:14.389472 2111 leader_election.cc:302] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 8abd1ea6dd3a4a0fa3d2e663327e23c5, 92f899461d3842f8bea6096a5f51a089, f447c357a6294e22a40921d7f03a83e0; no voters:
I20260619 06:49:14.389829 2379 raft_consensus.cc:2781] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:14.390295 2379 raft_consensus.cc:686] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [term 1 LEADER]: Becoming Leader. State: Replica: 92f899461d3842f8bea6096a5f51a089, State: Running, Role: LEADER
I20260619 06:49:14.390786 2379 consensus_queue.cc:237] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abd1ea6dd3a4a0fa3d2e663327e23c5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 34731 } } peers { permanent_uuid: "f447c357a6294e22a40921d7f03a83e0" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35377 } } peers { permanent_uuid: "92f899461d3842f8bea6096a5f51a089" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36955 } }
I20260619 06:49:14.395664 1906 catalog_manager.cc:5495] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92f899461d3842f8bea6096a5f51a089 (127.16.79.66). New cstate: current_term: 1 leader_uuid: "92f899461d3842f8bea6096a5f51a089" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8abd1ea6dd3a4a0fa3d2e663327e23c5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 34731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f447c357a6294e22a40921d7f03a83e0" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92f899461d3842f8bea6096a5f51a089" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36955 } health_report { overall_health: HEALTHY } } }
I20260619 06:49:14.455768 2370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:14.449923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34426 (local address 127.16.79.66:36955)
0619 06:49:14.450154 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:14.450161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:14.450285 (+ 124us) server_negotiation.cc:407] Connection header received
0619 06:49:14.450533 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:14.450537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:14.450645 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:14.450804 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:49:14.451169 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.451865 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.453520 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.453953 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.454089 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:14.454633 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:49:14.455152 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:49:14.455316 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:14.455396 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 06:49:14.455479 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:49:14.464442 2047 raft_consensus.cc:1261] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0 [term 1 FOLLOWER]: Refusing update from remote peer 92f899461d3842f8bea6096a5f51a089: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:49:14.464464 2308 raft_consensus.cc:1261] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5 [term 1 FOLLOWER]: Refusing update from remote peer 92f899461d3842f8bea6096a5f51a089: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:49:14.465456 2382 consensus_queue.cc:1035] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8abd1ea6dd3a4a0fa3d2e663327e23c5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 34731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:49:14.465811 2379 consensus_queue.cc:1035] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f447c357a6294e22a40921d7f03a83e0" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 35377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:49:14.496359 1906 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:49:14.496649 1906 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:49:14.500880 1906 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6650ccc8034488bab3ec4e936f085cf: Sending DeleteTablet for 3 replicas of tablet dcc65cd4f9144960b2683ba5bdb33e51
I20260619 06:49:14.502152 2027 tablet_service.cc:1514] Processing DeleteTablet for tablet dcc65cd4f9144960b2683ba5bdb33e51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:14 UTC) from {username='slave'} at 127.0.0.1:52456
I20260619 06:49:14.502152 2288 tablet_service.cc:1514] Processing DeleteTablet for tablet dcc65cd4f9144960b2683ba5bdb33e51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:14 UTC) from {username='slave'} at 127.0.0.1:52656
I20260619 06:49:14.505261 2158 tablet_service.cc:1514] Processing DeleteTablet for tablet dcc65cd4f9144960b2683ba5bdb33e51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:14 UTC) from {username='slave'} at 127.0.0.1:34422
I20260619 06:49:14.511190 2397 tablet_replica.cc:330] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089: stopping tablet replica
I20260619 06:49:14.511754 2397 raft_consensus.cc:2229] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:49:14.512651 2397 raft_consensus.cc:2258] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:14.513480 2396 tablet_replica.cc:330] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0: stopping tablet replica
I20260619 06:49:14.513945 2396 raft_consensus.cc:2229] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:49:14.514484 2396 raft_consensus.cc:2258] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:14.515220 2395 tablet_replica.cc:330] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5: stopping tablet replica
I20260619 06:49:14.515672 2395 raft_consensus.cc:2229] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:49:14.516191 2395 raft_consensus.cc:2258] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:14.519001 2397 ts_tablet_manager.cc:1905] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:14.519169 2395 ts_tablet_manager.cc:1905] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:14.519191 2396 ts_tablet_manager.cc:1905] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:14.521414 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 1965
I20260619 06:49:14.526640 2396 ts_tablet_manager.cc:1918] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:14.526890 2396 log.cc:1196] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/dcc65cd4f9144960b2683ba5bdb33e51
I20260619 06:49:14.527473 2396 ts_tablet_manager.cc:1939] T dcc65cd4f9144960b2683ba5bdb33e51 P f447c357a6294e22a40921d7f03a83e0: Deleting consensus metadata
I20260619 06:49:14.529611 1893 catalog_manager.cc:4841] TS f447c357a6294e22a40921d7f03a83e0 (127.16.79.65:35377): tablet dcc65cd4f9144960b2683ba5bdb33e51 (table test-table [id=608ac552c9e944eca4872c57570e7606]) successfully deleted
I20260619 06:49:14.533653 2395 ts_tablet_manager.cc:1918] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:14.533844 2395 log.cc:1196] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/dcc65cd4f9144960b2683ba5bdb33e51
I20260619 06:49:14.534361 2395 ts_tablet_manager.cc:1939] T dcc65cd4f9144960b2683ba5bdb33e51 P 8abd1ea6dd3a4a0fa3d2e663327e23c5: Deleting consensus metadata
I20260619 06:49:14.535358 2397 ts_tablet_manager.cc:1918] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:14.535535 2397 log.cc:1196] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/dcc65cd4f9144960b2683ba5bdb33e51
I20260619 06:49:14.536056 2397 ts_tablet_manager.cc:1939] T dcc65cd4f9144960b2683ba5bdb33e51 P 92f899461d3842f8bea6096a5f51a089: Deleting consensus metadata
I20260619 06:49:14.536368 1890 catalog_manager.cc:4841] TS 8abd1ea6dd3a4a0fa3d2e663327e23c5 (127.16.79.67:34731): tablet dcc65cd4f9144960b2683ba5bdb33e51 (table test-table [id=608ac552c9e944eca4872c57570e7606]) successfully deleted
I20260619 06:49:14.537674 1891 catalog_manager.cc:4841] TS 92f899461d3842f8bea6096a5f51a089 (127.16.79.66:36955): tablet dcc65cd4f9144960b2683ba5bdb33e51 (table test-table [id=608ac552c9e944eca4872c57570e7606]) successfully deleted
I20260619 06:49:14.540843 2368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:14.522584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35118 (local address 127.16.79.65:35377)
0619 06:49:14.522788 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:14.522796 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:14.528454 (+ 5658us) server_negotiation.cc:407] Connection header received
0619 06:49:14.528759 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:14.528777 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:14.528877 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:14.528944 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:14.530336 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.531455 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.536742 (+ 5287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.537393 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.537509 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:14.539126 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:14.539142 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:14.539164 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:14.539935 (+ 771us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:14.539995 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:14.540019 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:14.540164 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:14.540343 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 06:49:14.540465 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 06:49:14.541643 2087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:14.636281 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 1965
I20260619 06:49:14.654698 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 2096
I20260619 06:49:14.662447 2370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:14.655766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33602 (local address 127.16.79.66:36955)
0619 06:49:14.655978 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:14.655982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:14.655999 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:49:14.656051 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:14.656054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:14.656190 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:14.656335 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:14.656875 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.657767 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.659077 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.659534 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.659677 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:14.660963 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:14.660972 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:14.660987 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:14.661612 (+ 625us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:14.661720 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:14.661749 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:14.661879 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:14.662002 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 06:49:14.662106 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 06:49:14.663162 2218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:14.777885 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 2096
I20260619 06:49:14.797184 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 2226
I20260619 06:49:14.804476 2367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:14.797975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56614 (local address 127.16.79.67:34731)
0619 06:49:14.798164 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:14.798171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:14.798311 (+ 140us) server_negotiation.cc:407] Connection header received
0619 06:49:14.798536 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:14.798540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:14.798620 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:14.798733 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:14.799185 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.799902 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.801307 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:14.801722 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:14.801860 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:14.802949 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:14.802958 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:14.802974 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:14.803551 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:14.803606 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:14.803626 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:14.803779 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:14.803920 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 06:49:14.804033 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:49:14.805151 2348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:14.894817 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 2226
I20260619 06:49:14.910771 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 1875
I20260619 06:49:14.911972 1936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:15.029145 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 1875
2026-06-19T06:49:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3062 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:15 dist-test-slave-5w2x krb5kdc[2405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:15 dist-test-slave-5w2x krb5kdc[2405](info): set up 2 sockets
Jun 19 06:49:15 dist-test-slave-5w2x krb5kdc[2405](info): commencing operation
krb5kdc: starting...
W20260619 06:49:16.140599 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:16 dist-test-slave-5w2x krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:49:16.391510 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:37089
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:37089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:16.727993 2418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:16.728359 2418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:16.728516 2418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:16.735751 2418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:16.735860 2418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:16.735980 2418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:16.736078 2418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:16.736164 2418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:16.746459 2418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:37089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:37089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:16.747845 2418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:16.749276 2418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:16.761296 2423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:16.761377 2426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:16.761310 2424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:16.762847 2418 server_base.cc:1034] running on GCE node
I20260619 06:49:16.763701 2418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:16.765276 2418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:16.766543 2418 hybrid_clock.cc:648] HybridClock initialized: now 1781851756766495 us; error 46 us; skew 500 ppm
I20260619 06:49:16.766945 2418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:16 dist-test-slave-5w2x krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851756, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:16.770181 2418 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:49:16.771735 2418 webserver.cc:460] Webserver started at http://127.16.79.126:33791/ using document root <none> and password file <none>
I20260619 06:49:16.772512 2418 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:16.772670 2418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:16.772977 2418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:16.776024 2418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "69f50b27cb904e2997d3d55f829686ff"
format_stamp: "Formatted at 2026-06-19 06:49:16 on dist-test-slave-5w2x"
server_key: "c02c0190c15a5aceb0b6912618c429bc"
server_key_iv: "cbcb0e0edb1942af9d373b9f86d1b9e5"
server_key_version: "encryptionkey@0"
I20260619 06:49:16.776854 2418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "69f50b27cb904e2997d3d55f829686ff"
format_stamp: "Formatted at 2026-06-19 06:49:16 on dist-test-slave-5w2x"
server_key: "c02c0190c15a5aceb0b6912618c429bc"
server_key_iv: "cbcb0e0edb1942af9d373b9f86d1b9e5"
server_key_version: "encryptionkey@0"
I20260619 06:49:16.782435 2418 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:49:16.786211 2433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:16.787599 2418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:49:16.787811 2418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "69f50b27cb904e2997d3d55f829686ff"
format_stamp: "Formatted at 2026-06-19 06:49:16 on dist-test-slave-5w2x"
server_key: "c02c0190c15a5aceb0b6912618c429bc"
server_key_iv: "cbcb0e0edb1942af9d373b9f86d1b9e5"
server_key_version: "encryptionkey@0"
I20260619 06:49:16.788026 2418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:16.829398 2418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:16.834156 2418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:16.834478 2418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:16.854039 2418 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:37089
I20260619 06:49:16.854058 2484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:37089 every 8 connection(s)
I20260619 06:49:16.855537 2418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:16.856680 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 2418
I20260619 06:49:16.856886 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:16.857458 16701 external_mini_cluster.cc:1428] Setting key ea062bbaeb7070e49a9cbb0c32ee0396
I20260619 06:49:16.860461 2485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 06:49:16 dist-test-slave-5w2x krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:49:16.874198 2485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff: Bootstrap starting.
I20260619 06:49:16.877934 2485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:16.879348 2485 log.cc:824] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:16.881963 2488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:16.859413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34684 (local address 127.16.79.126:37089)
0619 06:49:16.860284 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:16.860297 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:16.860340 (+ 43us) server_negotiation.cc:407] Connection header received
0619 06:49:16.861998 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:16.862019 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:16.862408 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:16.862741 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:16.864590 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:16.865993 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:16.867874 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:16.868658 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:16.868792 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:16.873940 (+ 5148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:16.873948 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:16.873964 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:16.874009 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:16.877458 (+ 3449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:16.878204 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:16.878216 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:16.878226 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:16.878485 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:16.878873 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:16.878878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:16.878881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:16.879672 (+ 791us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:16.880016 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:16.880467 (+ 451us) server_negotiation.cc:299] Negotiation successful
0619 06:49:16.880882 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":358,"threads_started":1}
I20260619 06:49:16.882709 2485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff: No bootstrap required, opened a new log
I20260619 06:49:16.891099 2485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f50b27cb904e2997d3d55f829686ff" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37089 } }
I20260619 06:49:16.891574 2485 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:16.891713 2485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69f50b27cb904e2997d3d55f829686ff, State: Initialized, Role: FOLLOWER
I20260619 06:49:16.892331 2485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f50b27cb904e2997d3d55f829686ff" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37089 } }
I20260619 06:49:16.892637 2485 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:16.892791 2485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:16.892989 2485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:16.895385 2485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f50b27cb904e2997d3d55f829686ff" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37089 } }
I20260619 06:49:16.895939 2485 leader_election.cc:302] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69f50b27cb904e2997d3d55f829686ff; no voters:
I20260619 06:49:16.896607 2485 leader_election.cc:288] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:16.896772 2490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:16.897946 2490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [term 1 LEADER]: Becoming Leader. State: Replica: 69f50b27cb904e2997d3d55f829686ff, State: Running, Role: LEADER
I20260619 06:49:16.898484 2490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f50b27cb904e2997d3d55f829686ff" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37089 } }
I20260619 06:49:16.898752 2485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:16.901734 2491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69f50b27cb904e2997d3d55f829686ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f50b27cb904e2997d3d55f829686ff" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37089 } } }
I20260619 06:49:16.902240 2491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:16.902297 2492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69f50b27cb904e2997d3d55f829686ff. Latest consensus state: current_term: 1 leader_uuid: "69f50b27cb904e2997d3d55f829686ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f50b27cb904e2997d3d55f829686ff" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37089 } } }
I20260619 06:49:16.902592 2492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:16.904964 2497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:16.910904 2497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:16.920523 2497 catalog_manager.cc:1347] Generated new cluster ID: a48472ac1c294481af9725afc275798b
I20260619 06:49:16.920665 2497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:16.950457 2497 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:16.951929 2497 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:16.967934 2497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff: Generated new TSK 0
I20260619 06:49:16.968909 2497 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:49:17.033421 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:37089
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:17.382783 2511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:17.383134 2511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:17.383309 2511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:17.391016 2511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:17.391201 2511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:17.391417 2511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:49:17.401795 2511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:37089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:17.403175 2511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:17.404619 2511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:17.418527 2516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:17.418466 2519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:17.418502 2517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:17.419826 2511 server_base.cc:1034] running on GCE node
I20260619 06:49:17.420639 2511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:17.421737 2511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:17.422995 2511 hybrid_clock.cc:648] HybridClock initialized: now 1781851757422918 us; error 70 us; skew 500 ppm
I20260619 06:49:17.423367 2511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:17 dist-test-slave-5w2x krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851757, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:17.426891 2511 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:49:17.428535 2511 webserver.cc:460] Webserver started at http://127.16.79.65:34577/ using document root <none> and password file <none>
I20260619 06:49:17.429306 2511 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:17.429462 2511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:17.429826 2511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:17.433056 2511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "233a538786e84ef88f6e404a9b5c0c9f"
format_stamp: "Formatted at 2026-06-19 06:49:17 on dist-test-slave-5w2x"
server_key: "da57c686e6c1cd7db70d431517f86b09"
server_key_iv: "804ba54a0c659be67c8dea11c2f27197"
server_key_version: "encryptionkey@0"
I20260619 06:49:17.433873 2511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "233a538786e84ef88f6e404a9b5c0c9f"
format_stamp: "Formatted at 2026-06-19 06:49:17 on dist-test-slave-5w2x"
server_key: "da57c686e6c1cd7db70d431517f86b09"
server_key_iv: "804ba54a0c659be67c8dea11c2f27197"
server_key_version: "encryptionkey@0"
I20260619 06:49:17.439517 2511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:49:17.443367 2526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:17.444909 2511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:49:17.445115 2511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "233a538786e84ef88f6e404a9b5c0c9f"
format_stamp: "Formatted at 2026-06-19 06:49:17 on dist-test-slave-5w2x"
server_key: "da57c686e6c1cd7db70d431517f86b09"
server_key_iv: "804ba54a0c659be67c8dea11c2f27197"
server_key_version: "encryptionkey@0"
I20260619 06:49:17.445346 2511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:17.477231 2511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:17.482045 2511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:17.482398 2511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:17.483465 2511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:17.484928 2511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:17.485006 2511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:17.485208 2511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:17.485261 2511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:17.520393 2511 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:37587
I20260619 06:49:17.520438 2638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:37587 every 8 connection(s)
I20260619 06:49:17.521790 2511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:49:17.531940 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 2511
I20260619 06:49:17.532151 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:49:17.532639 16701 external_mini_cluster.cc:1428] Setting key f07decacccebe7579d27693f3dd24123
Jun 19 06:49:17 dist-test-slave-5w2x krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851757, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:49:17.542923 2641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:17.524618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:38915 (local address 127.16.79.126:37089)
0619 06:49:17.525089 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:17.525094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:17.526044 (+ 950us) server_negotiation.cc:407] Connection header received
0619 06:49:17.527955 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:17.527961 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:17.528060 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:17.528208 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:17.529810 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:17.530548 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:17.532258 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:17.532693 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:17.532833 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:17.537028 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:17.537033 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:17.537036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:17.537069 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:17.539660 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:17.540499 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:17.540505 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:17.540510 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:17.540647 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:17.541092 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:17.541097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:17.541104 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:17.541386 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:17.541540 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:17.542304 (+ 764us) server_negotiation.cc:299] Negotiation successful
0619 06:49:17.542546 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":241,"threads_started":1}
I20260619 06:49:17.543984 2642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:17.525147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:37089 (local address 127.16.79.65:38915)
0619 06:49:17.525870 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:17.525908 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:17.527677 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:17.528465 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:17.528474 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:17.529006 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:17.529621 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:17.529639 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:17.530727 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:17.530731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:17.532056 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:17.532079 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:17.532991 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:17.533000 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:17.533344 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:17.534135 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:49:17.534188 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:17.536793 (+ 2605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:17.539817 (+ 3024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:17.539827 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:17.539846 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:17.540377 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:17.540762 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:17.540766 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:17.540770 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:17.540964 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:17.541525 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:17.541533 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:17.541892 (+ 359us) client_negotiation.cc:769] Sending connection context
0619 06:49:17.542189 (+ 297us) client_negotiation.cc:240] Negotiation successful
0619 06:49:17.542751 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":289,"threads_started":1}
I20260619 06:49:17.545883 2639 heartbeater.cc:347] Connected to a master server at 127.16.79.126:37089
I20260619 06:49:17.546253 2639 heartbeater.cc:464] Registering TS with master...
I20260619 06:49:17.547029 2639 heartbeater.cc:511] Master 127.16.79.126:37089 requested a full tablet report, sending...
I20260619 06:49:17.549463 2450 ts_manager.cc:194] Registered new tserver with Master: 233a538786e84ef88f6e404a9b5c0c9f (127.16.79.65:37587)
I20260619 06:49:17.550962 2450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:38915
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:49:17.587378 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:37089
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:17.927173 2645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:17.927457 2645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:17.927552 2645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:17.934896 2645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:17.935019 2645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:17.935134 2645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:49:17.945587 2645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:37089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:17.946959 2645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:17.948240 2645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:17.962041 2651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:17.962132 2653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:17.962173 2650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:17.962909 2645 server_base.cc:1034] running on GCE node
I20260619 06:49:17.963694 2645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:17.964812 2645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:17.966038 2645 hybrid_clock.cc:648] HybridClock initialized: now 1781851757965961 us; error 67 us; skew 500 ppm
I20260619 06:49:17.966370 2645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:17 dist-test-slave-5w2x krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851757, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:17.969839 2645 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:49:17.971416 2645 webserver.cc:460] Webserver started at http://127.16.79.66:33297/ using document root <none> and password file <none>
I20260619 06:49:17.972070 2645 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:17.972246 2645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:17.972582 2645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:17.975534 2645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "b635d6fc1ccb45c4883b4bb49c164d3b"
format_stamp: "Formatted at 2026-06-19 06:49:17 on dist-test-slave-5w2x"
server_key: "c9deb942b3e4f4b00e63858e439cf3fc"
server_key_iv: "38223ee44d4fe2045f7456195c03e0b7"
server_key_version: "encryptionkey@0"
I20260619 06:49:17.976478 2645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "b635d6fc1ccb45c4883b4bb49c164d3b"
format_stamp: "Formatted at 2026-06-19 06:49:17 on dist-test-slave-5w2x"
server_key: "c9deb942b3e4f4b00e63858e439cf3fc"
server_key_iv: "38223ee44d4fe2045f7456195c03e0b7"
server_key_version: "encryptionkey@0"
I20260619 06:49:17.981709 2645 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:49:17.985337 2660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:17.986734 2645 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:49:17.986874 2645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "b635d6fc1ccb45c4883b4bb49c164d3b"
format_stamp: "Formatted at 2026-06-19 06:49:17 on dist-test-slave-5w2x"
server_key: "c9deb942b3e4f4b00e63858e439cf3fc"
server_key_iv: "38223ee44d4fe2045f7456195c03e0b7"
server_key_version: "encryptionkey@0"
I20260619 06:49:17.987051 2645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:18.010015 2645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:18.014338 2645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:18.014573 2645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:18.015475 2645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:18.016829 2645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:18.016908 2645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:18.016991 2645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:18.017035 2645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:18.051990 2645 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:36509
I20260619 06:49:18.052078 2772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:36509 every 8 connection(s)
I20260619 06:49:18.053367 2645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:49:18.054307 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 2645
I20260619 06:49:18.054508 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:49:18.054927 16701 external_mini_cluster.cc:1428] Setting key e3f4936899cede9a2449afa469b6d9d6
Jun 19 06:49:18 dist-test-slave-5w2x krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851757, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:49:18.080176 2775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.056356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:35743 (local address 127.16.79.126:37089)
0619 06:49:18.056921 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:18.056930 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:18.058228 (+ 1298us) server_negotiation.cc:407] Connection header received
0619 06:49:18.061597 (+ 3369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:18.061601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:18.061675 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:18.061783 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:18.064246 (+ 2463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.065014 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.067840 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.068671 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.068836 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.072716 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:18.072726 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:18.072735 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:18.072783 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:18.076700 (+ 3917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:18.077531 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:18.077540 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:18.077546 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:18.077723 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:18.078291 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:18.078298 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:18.078302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:18.078603 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:18.078767 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:18.079514 (+ 747us) server_negotiation.cc:299] Negotiation successful
0619 06:49:18.079752 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":292,"threads_started":1}
I20260619 06:49:18.081375 2776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.057076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:37089 (local address 127.16.79.66:35743)
0619 06:49:18.058025 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:18.058079 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:18.061242 (+ 3163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:18.062097 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:18.062125 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:18.063023 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:18.063930 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:18.063962 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.065242 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.065251 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:18.067675 (+ 2424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:18.067688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.068910 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.068916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:18.069143 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.069923 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:49:18.069953 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:18.072452 (+ 2499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:18.076862 (+ 4410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:18.076869 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:18.076885 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:18.077393 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:18.077916 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:18.077920 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:18.077922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:18.078135 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:18.078753 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:18.078761 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:18.079116 (+ 355us) client_negotiation.cc:769] Sending connection context
0619 06:49:18.079397 (+ 281us) client_negotiation.cc:240] Negotiation successful
0619 06:49:18.079968 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":351,"threads_started":1}
I20260619 06:49:18.082944 2773 heartbeater.cc:347] Connected to a master server at 127.16.79.126:37089
I20260619 06:49:18.083289 2773 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
I20260619 06:49:18.084062 2773 heartbeater.cc:511] Master 127.16.79.126:37089 requested a full tablet report, sending...
I20260619 06:49:18.085649 2450 ts_manager.cc:194] Registered new tserver with Master: b635d6fc1ccb45c4883b4bb49c164d3b (127.16.79.66:36509)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
I20260619 06:49:18.086642 2450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:35743
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:49:18.115626 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:37089
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:18.451198 2779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:18.451498 2779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:18.451650 2779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:18.459286 2779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:18.459401 2779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:18.459575 2779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:49:18.469568 2779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:37089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:18.470985 2779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:18.472294 2779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:18.485630 2784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:18.485575 2787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:18.485755 2785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:18.486997 2779 server_base.cc:1034] running on GCE node
I20260619 06:49:18.487708 2779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:18.488893 2779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:18.490155 2779 hybrid_clock.cc:648] HybridClock initialized: now 1781851758490015 us; error 103 us; skew 500 ppm
I20260619 06:49:18.490518 2779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:18 dist-test-slave-5w2x krb5kdc[2405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851758, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:18.494411 2779 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:49:18.496025 2779 webserver.cc:460] Webserver started at http://127.16.79.67:46625/ using document root <none> and password file <none>
I20260619 06:49:18.496827 2779 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:18.496973 2779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:18.497326 2779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:18.500698 2779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "edd347f1c6914abbb017c34df8274aa5"
format_stamp: "Formatted at 2026-06-19 06:49:18 on dist-test-slave-5w2x"
server_key: "a47ecb3ba36d1c6433ce38d9cc50ecf4"
server_key_iv: "6faab7db54957de382a2d52bb64fa790"
server_key_version: "encryptionkey@0"
I20260619 06:49:18.501541 2779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "edd347f1c6914abbb017c34df8274aa5"
format_stamp: "Formatted at 2026-06-19 06:49:18 on dist-test-slave-5w2x"
server_key: "a47ecb3ba36d1c6433ce38d9cc50ecf4"
server_key_iv: "6faab7db54957de382a2d52bb64fa790"
server_key_version: "encryptionkey@0"
I20260619 06:49:18.507350 2779 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 06:49:18.510947 2794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:18.512384 2779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:49:18.512564 2779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "edd347f1c6914abbb017c34df8274aa5"
format_stamp: "Formatted at 2026-06-19 06:49:18 on dist-test-slave-5w2x"
server_key: "a47ecb3ba36d1c6433ce38d9cc50ecf4"
server_key_iv: "6faab7db54957de382a2d52bb64fa790"
server_key_version: "encryptionkey@0"
I20260619 06:49:18.512781 2779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:18.532732 2779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:18.537264 2779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:18.537566 2779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:18.538549 2779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:18.539934 2779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:18.540009 2779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:18.540212 2779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:18.540284 2779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:18.554219 2639 heartbeater.cc:503] Master 127.16.79.126:37089 was elected leader, sending a full tablet report...
I20260619 06:49:18.573876 2779 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:43705
I20260619 06:49:18.573949 2906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:43705 every 8 connection(s)
I20260619 06:49:18.575194 2779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:49:18.580785 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 2779
I20260619 06:49:18.580982 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:49:18.581583 16701 external_mini_cluster.cc:1428] Setting key 8e54e1118947364e19e412f3e67ac6de
Jun 19 06:49:18 dist-test-slave-5w2x krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851758, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:49:18.597500 2775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.577871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:49543 (local address 127.16.79.126:37089)
0619 06:49:18.578056 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:18.578061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:18.579380 (+ 1319us) server_negotiation.cc:407] Connection header received
0619 06:49:18.581756 (+ 2376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:18.581760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:18.581838 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:18.581922 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:18.583759 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.584869 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.586417 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.586830 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.586927 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.590819 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:18.590824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:18.590828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:18.590867 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:18.593896 (+ 3029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:18.594772 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:18.594777 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:18.594779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:18.594963 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:18.595489 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:18.595495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:18.595500 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:18.595836 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:18.596001 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:18.596877 (+ 876us) server_negotiation.cc:299] Negotiation successful
0619 06:49:18.597122 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 06:49:18.598578 2909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.578461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:37089 (local address 127.16.79.67:49543)
0619 06:49:18.579199 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:18.579242 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:18.581372 (+ 2130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:18.582245 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:18.582257 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:18.582886 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:18.583598 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:18.583616 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.585006 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.585010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:18.586279 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:18.586292 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.587043 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.587047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:18.587259 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.587889 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:49:18.587916 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:18.590540 (+ 2624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:18.594080 (+ 3540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:18.594089 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:18.594104 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:18.594633 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:18.595089 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:18.595094 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:18.595098 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:18.595362 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:18.595973 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:18.595982 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:18.596513 (+ 531us) client_negotiation.cc:769] Sending connection context
0619 06:49:18.596802 (+ 289us) client_negotiation.cc:240] Negotiation successful
0619 06:49:18.597394 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":283,"threads_started":1}
I20260619 06:49:18.600103 2907 heartbeater.cc:347] Connected to a master server at 127.16.79.126:37089
I20260619 06:49:18.600519 2907 heartbeater.cc:464] Registering TS with master...
I20260619 06:49:18.601293 2907 heartbeater.cc:511] Master 127.16.79.126:37089 requested a full tablet report, sending...
I20260619 06:49:18.602798 2450 ts_manager.cc:194] Registered new tserver with Master: edd347f1c6914abbb017c34df8274aa5 (127.16.79.67:43705)
I20260619 06:49:18.603647 2450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:49543
I20260619 06:49:18.609723 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:49:18.628087 2775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.613949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34686 (local address 127.16.79.126:37089)
0619 06:49:18.614157 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:18.614162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:18.614779 (+ 617us) server_negotiation.cc:407] Connection header received
0619 06:49:18.614921 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:18.614925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:18.614990 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:18.615084 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:18.616734 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.617610 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.619051 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.619499 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.619607 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.622418 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:18.622423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:18.622427 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:18.622468 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:18.625175 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:18.625973 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:18.625977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:18.625979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:18.626152 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:18.626546 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:18.626548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:18.626550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:18.626853 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:18.627054 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:18.627587 (+ 533us) server_negotiation.cc:299] Negotiation successful
0619 06:49:18.627784 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 06:49:18.632194 2450 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:49:18.639191 2450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:49:18.672809 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.652628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:37587 (local address 127.0.0.1:39108)
0619 06:49:18.653503 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:18.653561 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:18.653753 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:18.657401 (+ 3648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:18.657415 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:18.657467 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:18.658131 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:18.658152 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.663219 (+ 5067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.663229 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:18.668949 (+ 5720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:18.668977 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.671657 (+ 2680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.671668 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:18.671930 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.672006 (+ 76us) client_negotiation.cc:769] Sending connection context
0619 06:49:18.672141 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 06:49:18.672395 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":408,"threads_started":1}
I20260619 06:49:18.673251 2920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.653212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39108 (local address 127.16.79.65:37587)
0619 06:49:18.654457 (+ 1245us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:18.654470 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:18.654505 (+ 35us) server_negotiation.cc:407] Connection header received
0619 06:49:18.654853 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:18.654867 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:18.655261 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:18.658373 (+ 3112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:18.658716 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.662961 (+ 4245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.669237 (+ 6276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.671289 (+ 2052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.672299 (+ 1010us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.672585 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:18.672674 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 06:49:18.672878 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1029,"thread_start_us":1573,"threads_started":1}
I20260619 06:49:18.677423 2574 tablet_service.cc:1467] Processing CreateTablet for tablet 518a023e2f3245388772fc2c01dd6cc4 (DEFAULT_TABLE table=test-table [id=68807be6eefb4311bf56666d805a1422]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:18.679303 2574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518a023e2f3245388772fc2c01dd6cc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:18.679754 2921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.654166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:36509 (local address 127.0.0.1:36310)
0619 06:49:18.666994 (+ 12828us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:18.667024 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:18.667215 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:18.670888 (+ 3673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:18.670893 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:18.670941 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:18.671817 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:18.671832 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.673967 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.673976 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:18.676668 (+ 2692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:18.676693 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.678714 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.678720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:18.678906 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.678946 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 06:49:18.679074 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 06:49:18.679251 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12623,"mutex_wait_us":133,"spinlock_wait_cycles":1280,"thread_start_us":12446,"threads_started":1}
I20260619 06:49:18.680428 2924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.654684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36310 (local address 127.16.79.66:36509)
0619 06:49:18.670057 (+ 15373us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:18.670066 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:18.670091 (+ 25us) server_negotiation.cc:407] Connection header received
0619 06:49:18.670212 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:18.670219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:18.670436 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:18.670620 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:18.672074 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.673810 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.676899 (+ 3089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.678461 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.678620 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.678847 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:18.679887 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0619 06:49:18.680072 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15207,"thread_start_us":15090,"threads_started":1}
I20260619 06:49:18.680929 2922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.654303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:43705 (local address 127.0.0.1:49102)
0619 06:49:18.667964 (+ 13661us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:18.667989 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:18.668129 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:18.670887 (+ 2758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:18.670893 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:18.670947 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:18.671824 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:18.671834 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.674214 (+ 2380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.674220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:18.676699 (+ 2479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:18.676710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.679090 (+ 2380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.679097 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:18.679315 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.679361 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 06:49:18.679456 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 06:49:18.679606 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13469,"mutex_wait_us":48,"spinlock_wait_cycles":4480,"thread_start_us":13218,"threads_started":1}
I20260619 06:49:18.681749 2923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.654493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49102 (local address 127.16.79.67:43705)
0619 06:49:18.669785 (+ 15292us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:18.669798 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:18.669841 (+ 43us) server_negotiation.cc:407] Connection header received
0619 06:49:18.669957 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:18.669967 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:18.670339 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:18.670591 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:18.672283 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.674061 (+ 1778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.677118 (+ 3057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.678576 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.678702 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.680366 (+ 1664us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:18.680436 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 06:49:18.681381 (+ 945us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15063,"thread_start_us":14889,"threads_started":1}
I20260619 06:49:18.684448 2708 tablet_service.cc:1467] Processing CreateTablet for tablet 518a023e2f3245388772fc2c01dd6cc4 (DEFAULT_TABLE table=test-table [id=68807be6eefb4311bf56666d805a1422]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:18.685873 2842 tablet_service.cc:1467] Processing CreateTablet for tablet 518a023e2f3245388772fc2c01dd6cc4 (DEFAULT_TABLE table=test-table [id=68807be6eefb4311bf56666d805a1422]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:18.686760 2708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518a023e2f3245388772fc2c01dd6cc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:18.687528 2842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518a023e2f3245388772fc2c01dd6cc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:18.694876 2925 tablet_bootstrap.cc:492] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f: Bootstrap starting.
I20260619 06:49:18.698295 2925 tablet_bootstrap.cc:654] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:18.699647 2925 log.cc:824] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:18.705268 2927 tablet_bootstrap.cc:492] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5: Bootstrap starting.
I20260619 06:49:18.707309 2928 tablet_bootstrap.cc:492] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b: Bootstrap starting.
I20260619 06:49:18.708626 2925 tablet_bootstrap.cc:492] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f: No bootstrap required, opened a new log
I20260619 06:49:18.708940 2925 ts_tablet_manager.cc:1397] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.011s
I20260619 06:49:18.710431 2927 tablet_bootstrap.cc:654] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:18.710600 2928 tablet_bootstrap.cc:654] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:18.711813 2928 log.cc:824] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:18.711844 2927 log.cc:824] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:18.715385 2928 tablet_bootstrap.cc:492] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b: No bootstrap required, opened a new log
I20260619 06:49:18.715385 2927 tablet_bootstrap.cc:492] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5: No bootstrap required, opened a new log
I20260619 06:49:18.715690 2927 ts_tablet_manager.cc:1397] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260619 06:49:18.715698 2928 ts_tablet_manager.cc:1397] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260619 06:49:18.718962 2925 raft_consensus.cc:348] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233a538786e84ef88f6e404a9b5c0c9f" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37587 } } peers { permanent_uuid: "b635d6fc1ccb45c4883b4bb49c164d3b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36509 } } peers { permanent_uuid: "edd347f1c6914abbb017c34df8274aa5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43705 } }
I20260619 06:49:18.719632 2925 raft_consensus.cc:374] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:18.719821 2925 raft_consensus.cc:729] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233a538786e84ef88f6e404a9b5c0c9f, State: Initialized, Role: FOLLOWER
I20260619 06:49:18.720542 2925 consensus_queue.cc:260] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233a538786e84ef88f6e404a9b5c0c9f" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37587 } } peers { permanent_uuid: "b635d6fc1ccb45c4883b4bb49c164d3b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36509 } } peers { permanent_uuid: "edd347f1c6914abbb017c34df8274aa5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43705 } }
I20260619 06:49:18.722507 2925 ts_tablet_manager.cc:1428] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260619 06:49:18.728029 2927 raft_consensus.cc:348] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233a538786e84ef88f6e404a9b5c0c9f" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37587 } } peers { permanent_uuid: "b635d6fc1ccb45c4883b4bb49c164d3b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36509 } } peers { permanent_uuid: "edd347f1c6914abbb017c34df8274aa5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43705 } }
I20260619 06:49:18.728029 2928 raft_consensus.cc:348] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233a538786e84ef88f6e404a9b5c0c9f" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37587 } } peers { permanent_uuid: "b635d6fc1ccb45c4883b4bb49c164d3b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36509 } } peers { permanent_uuid: "edd347f1c6914abbb017c34df8274aa5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43705 } }
I20260619 06:49:18.728585 2927 raft_consensus.cc:374] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:18.728585 2928 raft_consensus.cc:374] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:18.728708 2928 raft_consensus.cc:729] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b635d6fc1ccb45c4883b4bb49c164d3b, State: Initialized, Role: FOLLOWER
I20260619 06:49:18.728709 2927 raft_consensus.cc:729] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edd347f1c6914abbb017c34df8274aa5, State: Initialized, Role: FOLLOWER
I20260619 06:49:18.729421 2927 consensus_queue.cc:260] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233a538786e84ef88f6e404a9b5c0c9f" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37587 } } peers { permanent_uuid: "b635d6fc1ccb45c4883b4bb49c164d3b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36509 } } peers { permanent_uuid: "edd347f1c6914abbb017c34df8274aa5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43705 } }
I20260619 06:49:18.729421 2928 consensus_queue.cc:260] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233a538786e84ef88f6e404a9b5c0c9f" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37587 } } peers { permanent_uuid: "b635d6fc1ccb45c4883b4bb49c164d3b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36509 } } peers { permanent_uuid: "edd347f1c6914abbb017c34df8274aa5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43705 } }
I20260619 06:49:18.730662 2773 heartbeater.cc:503] Master 127.16.79.126:37089 was elected leader, sending a full tablet report...
I20260619 06:49:18.731614 2928 ts_tablet_manager.cc:1428] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260619 06:49:18.731786 2927 ts_tablet_manager.cc:1428] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260619 06:49:18.732004 2907 heartbeater.cc:503] Master 127.16.79.126:37089 was elected leader, sending a full tablet report...
I20260619 06:49:18.772691 2933 raft_consensus.cc:482] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:49:18.773044 2933 raft_consensus.cc:504] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233a538786e84ef88f6e404a9b5c0c9f" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37587 } } peers { permanent_uuid: "b635d6fc1ccb45c4883b4bb49c164d3b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36509 } } peers { permanent_uuid: "edd347f1c6914abbb017c34df8274aa5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43705 } }
W20260619 06:49:18.774384 2640 tablet.cc:2369] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:49:18.774976 2933 leader_election.cc:288] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 233a538786e84ef88f6e404a9b5c0c9f (127.16.79.65:37587), b635d6fc1ccb45c4883b4bb49c164d3b (127.16.79.66:36509)
I20260619 06:49:18.781425 2909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.775335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:37587 (local address 127.16.79.67:58155)
0619 06:49:18.775533 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:18.775552 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:18.775704 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:18.776072 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:18.776077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:18.776107 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:18.776430 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:18.776443 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.777725 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.777728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:18.779527 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:18.779542 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.780866 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.780870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:18.780996 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.781020 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 06:49:18.781096 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 06:49:18.781221 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 06:49:18.781540 2920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.775335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:58155 (local address 127.16.79.65:37587)
0619 06:49:18.775520 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:18.775525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:18.775663 (+ 138us) server_negotiation.cc:407] Connection header received
0619 06:49:18.775855 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:18.775860 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:18.775925 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:18.776049 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:18.776609 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.777596 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.779698 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.780731 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.780891 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.780966 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:18.781154 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 06:49:18.781287 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 06:49:18.782578 2934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.775761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:36509 (local address 127.16.79.67:53935)
0619 06:49:18.776680 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:18.776698 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:18.776787 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:18.777078 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:18.777081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:18.777097 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:18.777373 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:18.777379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.778402 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.778408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:18.780376 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:18.780390 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.781956 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.781962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:18.782142 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.782179 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 06:49:18.782259 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 06:49:18.782346 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"thread_start_us":587,"threads_started":1}
I20260619 06:49:18.782712 2924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.776099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:53935 (local address 127.16.79.66:36509)
0619 06:49:18.776371 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:18.776377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:18.776781 (+ 404us) server_negotiation.cc:407] Connection header received
0619 06:49:18.776880 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:18.776887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:18.776961 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:18.777060 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:18.777514 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.778246 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.780539 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.781759 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.781928 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.781978 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:18.782323 (+ 345us) server_negotiation.cc:299] Negotiation successful
0619 06:49:18.782460 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260619 06:49:18.782711 2594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518a023e2f3245388772fc2c01dd6cc4" candidate_uuid: "edd347f1c6914abbb017c34df8274aa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233a538786e84ef88f6e404a9b5c0c9f" is_pre_election: true
I20260619 06:49:18.783306 2594 raft_consensus.cc:2454] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate edd347f1c6914abbb017c34df8274aa5 in term 0.
I20260619 06:49:18.783828 2728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518a023e2f3245388772fc2c01dd6cc4" candidate_uuid: "edd347f1c6914abbb017c34df8274aa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b635d6fc1ccb45c4883b4bb49c164d3b" is_pre_election: true
I20260619 06:49:18.784067 2795 leader_election.cc:302] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 233a538786e84ef88f6e404a9b5c0c9f, edd347f1c6914abbb017c34df8274aa5; no voters:
I20260619 06:49:18.784415 2728 raft_consensus.cc:2454] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate edd347f1c6914abbb017c34df8274aa5 in term 0.
I20260619 06:49:18.784550 2933 raft_consensus.cc:2781] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:49:18.784698 2933 raft_consensus.cc:482] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:49:18.784802 2933 raft_consensus.cc:3037] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:18.787127 2933 raft_consensus.cc:504] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233a538786e84ef88f6e404a9b5c0c9f" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37587 } } peers { permanent_uuid: "b635d6fc1ccb45c4883b4bb49c164d3b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36509 } } peers { permanent_uuid: "edd347f1c6914abbb017c34df8274aa5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43705 } }
I20260619 06:49:18.787765 2933 leader_election.cc:288] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [CANDIDATE]: Term 1 election: Requested vote from peers 233a538786e84ef88f6e404a9b5c0c9f (127.16.79.65:37587), b635d6fc1ccb45c4883b4bb49c164d3b (127.16.79.66:36509)
I20260619 06:49:18.788331 2728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518a023e2f3245388772fc2c01dd6cc4" candidate_uuid: "edd347f1c6914abbb017c34df8274aa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b635d6fc1ccb45c4883b4bb49c164d3b"
I20260619 06:49:18.788338 2594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518a023e2f3245388772fc2c01dd6cc4" candidate_uuid: "edd347f1c6914abbb017c34df8274aa5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "233a538786e84ef88f6e404a9b5c0c9f"
I20260619 06:49:18.788532 2728 raft_consensus.cc:3037] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:18.788550 2594 raft_consensus.cc:3037] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:18.790969 2728 raft_consensus.cc:2454] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate edd347f1c6914abbb017c34df8274aa5 in term 1.
I20260619 06:49:18.791028 2594 raft_consensus.cc:2454] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate edd347f1c6914abbb017c34df8274aa5 in term 1.
I20260619 06:49:18.791417 2795 leader_election.cc:302] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b635d6fc1ccb45c4883b4bb49c164d3b, edd347f1c6914abbb017c34df8274aa5; no voters:
I20260619 06:49:18.791862 2933 raft_consensus.cc:2781] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:18.792469 2933 raft_consensus.cc:686] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [term 1 LEADER]: Becoming Leader. State: Replica: edd347f1c6914abbb017c34df8274aa5, State: Running, Role: LEADER
I20260619 06:49:18.793046 2933 consensus_queue.cc:237] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233a538786e84ef88f6e404a9b5c0c9f" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37587 } } peers { permanent_uuid: "b635d6fc1ccb45c4883b4bb49c164d3b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36509 } } peers { permanent_uuid: "edd347f1c6914abbb017c34df8274aa5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43705 } }
I20260619 06:49:18.797991 2450 catalog_manager.cc:5495] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 reported cstate change: term changed from 0 to 1, leader changed from <none> to edd347f1c6914abbb017c34df8274aa5 (127.16.79.67). New cstate: current_term: 1 leader_uuid: "edd347f1c6914abbb017c34df8274aa5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "233a538786e84ef88f6e404a9b5c0c9f" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37587 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b635d6fc1ccb45c4883b4bb49c164d3b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edd347f1c6914abbb017c34df8274aa5" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43705 } health_report { overall_health: HEALTHY } } }
W20260619 06:49:18.805555 2774 tablet.cc:2369] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:49:18.822621 2923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.816911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49104 (local address 127.16.79.67:43705)
0619 06:49:18.817180 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:18.817186 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:18.817205 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:49:18.817416 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:18.817420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:18.817500 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:18.817656 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:49:18.818067 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.818812 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.820315 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.820720 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.820875 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.821495 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:49:18.822001 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:49:18.822209 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:18.822319 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 06:49:18.822365 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260619 06:49:18.826861 2908 tablet.cc:2369] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:49:18.831287 2728 raft_consensus.cc:1261] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b [term 1 FOLLOWER]: Refusing update from remote peer edd347f1c6914abbb017c34df8274aa5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:49:18.831319 2594 raft_consensus.cc:1261] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f [term 1 FOLLOWER]: Refusing update from remote peer edd347f1c6914abbb017c34df8274aa5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:49:18.832377 2935 consensus_queue.cc:1035] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b635d6fc1ccb45c4883b4bb49c164d3b" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 36509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:49:18.832815 2933 consensus_queue.cc:1035] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "233a538786e84ef88f6e404a9b5c0c9f" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 37587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:49:18.849824 2940 mvcc.cc:205] Tried to move back new op lower bound from 7298464804161822720 to 7298464804020047872. Current Snapshot: MvccSnapshot[applied={T|T < 7298464804161822720}]
I20260619 06:49:18.862207 2449 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:49:18.862522 2449 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:49:18.866226 2449 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69f50b27cb904e2997d3d55f829686ff: Sending DeleteTablet for 3 replicas of tablet 518a023e2f3245388772fc2c01dd6cc4
I20260619 06:49:18.867432 2842 tablet_service.cc:1514] Processing DeleteTablet for tablet 518a023e2f3245388772fc2c01dd6cc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:18 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:49102
I20260619 06:49:18.867426 2574 tablet_service.cc:1514] Processing DeleteTablet for tablet 518a023e2f3245388772fc2c01dd6cc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:18 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:39108
I20260619 06:49:18.868443 2708 tablet_service.cc:1514] Processing DeleteTablet for tablet 518a023e2f3245388772fc2c01dd6cc4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:18 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:36310
I20260619 06:49:18.881584 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 2511
I20260619 06:49:18.896029 2950 tablet_replica.cc:330] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b: stopping tablet replica
I20260619 06:49:18.896898 2950 raft_consensus.cc:2229] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:49:18.897612 2950 raft_consensus.cc:2258] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:18.898367 2948 tablet_replica.cc:330] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5: stopping tablet replica
I20260619 06:49:18.900396 2948 raft_consensus.cc:2229] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:49:18.902022 2950 ts_tablet_manager.cc:1905] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:18.903084 2948 raft_consensus.cc:2258] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:18.903769 2949 tablet_replica.cc:330] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f: stopping tablet replica
I20260619 06:49:18.904235 2949 raft_consensus.cc:2229] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:49:18.905035 2949 raft_consensus.cc:2258] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f [term 1 FOLLOWER]: Raft consensus is shut down!
Jun 19 06:49:18 dist-test-slave-5w2x krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:49:18.907712 2948 ts_tablet_manager.cc:1905] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:18.909417 2949 ts_tablet_manager.cc:1905] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:18.911904 2950 ts_tablet_manager.cc:1918] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:18.912166 2950 log.cc:1196] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/518a023e2f3245388772fc2c01dd6cc4
I20260619 06:49:18.912819 2950 ts_tablet_manager.cc:1939] T 518a023e2f3245388772fc2c01dd6cc4 P b635d6fc1ccb45c4883b4bb49c164d3b: Deleting consensus metadata
I20260619 06:49:18.914932 2434 catalog_manager.cc:4841] TS b635d6fc1ccb45c4883b4bb49c164d3b (127.16.79.66:36509): tablet 518a023e2f3245388772fc2c01dd6cc4 (table test-table [id=68807be6eefb4311bf56666d805a1422]) successfully deleted
I20260619 06:49:18.916111 2920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:18.883131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39110 (local address 127.16.79.65:37587)
0619 06:49:18.883330 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:18.883337 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:18.893876 (+ 10539us) server_negotiation.cc:407] Connection header received
0619 06:49:18.894405 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:18.894412 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:18.894507 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:18.894607 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:18.896606 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.897831 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.900026 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:18.900798 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:18.900913 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:18.907747 (+ 6834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:18.907768 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:18.907787 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:18.907841 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:18.911962 (+ 4121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:18.913106 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:18.913121 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:18.913125 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:18.913319 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:18.913871 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:18.913877 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:18.913882 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:18.914381 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:18.914720 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:18.915309 (+ 589us) server_negotiation.cc:299] Negotiation successful
0619 06:49:18.915580 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 06:49:18.916767 2948 ts_tablet_manager.cc:1918] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:18.916982 2948 log.cc:1196] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/518a023e2f3245388772fc2c01dd6cc4
I20260619 06:49:18.917718 2948 ts_tablet_manager.cc:1939] T 518a023e2f3245388772fc2c01dd6cc4 P edd347f1c6914abbb017c34df8274aa5: Deleting consensus metadata
I20260619 06:49:18.918212 2634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:18.919607 2436 catalog_manager.cc:4841] TS edd347f1c6914abbb017c34df8274aa5 (127.16.79.67:43705): tablet 518a023e2f3245388772fc2c01dd6cc4 (table test-table [id=68807be6eefb4311bf56666d805a1422]) successfully deleted
I20260619 06:49:19.011384 2949 ts_tablet_manager.cc:1918] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:19.011675 2949 log.cc:1196] T 518a023e2f3245388772fc2c01dd6cc4 P 233a538786e84ef88f6e404a9b5c0c9f: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/518a023e2f3245388772fc2c01dd6cc4
I20260619 06:49:19.012065 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 2511
W20260619 06:49:19.032117 2434 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.79.65:37587 (error 108)
I20260619 06:49:19.032327 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 2645
I20260619 06:49:19.034289 2922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:19.033810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:37587 (local address 127.0.0.1:39116)
0619 06:49:19.034000 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:19.034137 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260619 06:49:19.034576 2434 catalog_manager.cc:4584] TS 233a538786e84ef88f6e404a9b5c0c9f (127.16.79.65:37587): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 518a023e2f3245388772fc2c01dd6cc4: Network error: Client connection negotiation failed: client connection to 127.16.79.65:37587: connect: Connection refused (error 111)
Jun 19 06:49:19 dist-test-slave-5w2x krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:49:19.050633 2924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:19.033363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36318 (local address 127.16.79.66:36509)
0619 06:49:19.033543 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:19.033549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:19.033569 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:49:19.033727 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:19.033732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:19.033805 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:19.033918 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:19.035731 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:19.036616 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:19.038486 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:19.038866 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:19.038967 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:19.044107 (+ 5140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:19.044122 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:19.044317 (+ 195us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:19.044363 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:19.047280 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:19.048053 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:19.048060 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:19.048062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:19.048423 (+ 361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:19.048959 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:19.048964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:19.048969 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:19.049384 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:19.049703 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:19.050016 (+ 313us) server_negotiation.cc:299] Negotiation successful
0619 06:49:19.050255 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 06:49:19.051283 2768 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:19.088366 2922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:19.087716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:37587 (local address 127.0.0.1:39126)
0619 06:49:19.087982 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:19.088101 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260619 06:49:19.089592 2922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:19.089176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:37587 (local address 127.0.0.1:39142)
0619 06:49:19.089342 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:19.089453 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 06:49:19.143428 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 2645
I20260619 06:49:19.150529 2922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:19.149924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:37587 (local address 127.0.0.1:39158)
0619 06:49:19.150189 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:19.150307 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260619 06:49:19.151809 2922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:19.151365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:37587 (local address 127.0.0.1:39170)
0619 06:49:19.151554 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:19.151653 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 06:49:19.161115 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 2779
Jun 19 06:49:19 dist-test-slave-5w2x krb5kdc[2405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:49:19.180248 2923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:19.161884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49116 (local address 127.16.79.67:43705)
0619 06:49:19.162109 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:19.162121 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:19.162160 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:49:19.162234 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:19.162241 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:19.162350 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:19.162465 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:19.163765 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:19.165035 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:19.166632 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:19.167446 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:19.167610 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:19.172519 (+ 4909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:19.172540 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:19.172562 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:19.172625 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:19.177028 (+ 4403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:19.177828 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:19.177841 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:19.177847 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:19.178076 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:19.178536 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:19.178540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:19.178544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:19.179005 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:19.179333 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:19.179561 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 06:49:19.179797 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260619 06:49:19.180961 2902 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:19.232364 2922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:19.231808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:37587 (local address 127.0.0.1:39182)
0619 06:49:19.232046 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:19.232204 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260619 06:49:19.233511 2922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:19.233125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:37587 (local address 127.0.0.1:39186)
0619 06:49:19.233317 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:19.233402 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260619 06:49:19.313724 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 2779
I20260619 06:49:19.330741 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 2418
I20260619 06:49:19.331924 2480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:19.445556 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 2418
I20260619 06:49:19.445624 2922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:19.445115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:37587 (local address 127.0.0.1:39198)
0619 06:49:19.445313 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:19.445416 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
2026-06-19T06:49:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4417 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T06:49:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:19Z Disabled control of system clock
I20260619 06:49:19.523334 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:34315
--webserver_interface=127.16.79.126
--webserver_port=0
--builtin_ntp_servers=127.16.79.84:33269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:34315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 06:49:19.856081 2960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:19.856491 2960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:19.856639 2960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:19.864038 2960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:19.864219 2960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:19.864308 2960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:19.864399 2960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:19.864467 2960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:19.873947 2960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:33269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:34315
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:34315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:19.875334 2960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:19.877236 2960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:19.889210 2965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:19.889298 2966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:19.889292 2968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:19.890637 2960 server_base.cc:1034] running on GCE node
I20260619 06:49:19.891552 2960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:19.893263 2960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:19.894481 2960 hybrid_clock.cc:648] HybridClock initialized: now 1781851759894437 us; error 39 us; skew 500 ppm
I20260619 06:49:19.894889 2960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:49:19.897348 2960 webserver.cc:460] Webserver started at http://127.16.79.126:39203/ using document root <none> and password file <none>
I20260619 06:49:19.898082 2960 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:19.898177 2960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:19.898530 2960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:19.901659 2960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "bf73ef7400dd4d7c8a44fb470eb4cbd4"
format_stamp: "Formatted at 2026-06-19 06:49:19 on dist-test-slave-5w2x"
server_key: "47539adf255f29a310aca62e5288df38"
server_key_iv: "7d349e93e492b092fd560ceea0806ee5"
server_key_version: "encryptionkey@0"
I20260619 06:49:19.902531 2960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "bf73ef7400dd4d7c8a44fb470eb4cbd4"
format_stamp: "Formatted at 2026-06-19 06:49:19 on dist-test-slave-5w2x"
server_key: "47539adf255f29a310aca62e5288df38"
server_key_iv: "7d349e93e492b092fd560ceea0806ee5"
server_key_version: "encryptionkey@0"
I20260619 06:49:19.908262 2960 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:49:19.912021 2974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:19.913600 2960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:49:19.913801 2960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "bf73ef7400dd4d7c8a44fb470eb4cbd4"
format_stamp: "Formatted at 2026-06-19 06:49:19 on dist-test-slave-5w2x"
server_key: "47539adf255f29a310aca62e5288df38"
server_key_iv: "7d349e93e492b092fd560ceea0806ee5"
server_key_version: "encryptionkey@0"
I20260619 06:49:19.914045 2960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:19.941325 2960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:19.942286 2960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:19.942600 2960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:19.961871 2960 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:34315
I20260619 06:49:19.961895 3025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:34315 every 8 connection(s)
I20260619 06:49:19.963331 2960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:19.967343 3026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:19.968751 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 2960
I20260619 06:49:19.968950 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:19.969527 16701 external_mini_cluster.cc:1428] Setting key 6d79b0f50f7503893a868c0478a2f512
I20260619 06:49:19.980991 3026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4: Bootstrap starting.
I20260619 06:49:19.983448 3029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:19.971636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57184 (local address 127.16.79.126:34315)
0619 06:49:19.972816 (+ 1180us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:19.972827 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:19.972870 (+ 43us) server_negotiation.cc:407] Connection header received
0619 06:49:19.974544 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:19.974568 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:19.975075 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:19.975525 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:19.976091 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:19.977411 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:19.978787 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:19.979502 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:19.979713 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:19.981059 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:19.981088 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:19.981108 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:19.981695 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:19.981780 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:19.981802 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:19.981929 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:19.982161 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 06:49:19.982489 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":452,"threads_started":1}
I20260619 06:49:19.984465 3026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:19.985553 3026 log.cc:824] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:19.988441 3026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4: No bootstrap required, opened a new log
I20260619 06:49:19.995597 3026 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf73ef7400dd4d7c8a44fb470eb4cbd4" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34315 } }
I20260619 06:49:19.996018 3026 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:19.996126 3026 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf73ef7400dd4d7c8a44fb470eb4cbd4, State: Initialized, Role: FOLLOWER
I20260619 06:49:19.996724 3026 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf73ef7400dd4d7c8a44fb470eb4cbd4" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34315 } }
I20260619 06:49:19.996971 3026 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:19.997113 3026 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:19.997292 3026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:19.999498 3026 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf73ef7400dd4d7c8a44fb470eb4cbd4" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34315 } }
I20260619 06:49:20.000116 3026 leader_election.cc:302] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf73ef7400dd4d7c8a44fb470eb4cbd4; no voters:
I20260619 06:49:20.000844 3026 leader_election.cc:288] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:20.000993 3031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:20.003552 3026 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:20.004423 3031 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [term 1 LEADER]: Becoming Leader. State: Replica: bf73ef7400dd4d7c8a44fb470eb4cbd4, State: Running, Role: LEADER
I20260619 06:49:20.005204 3031 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf73ef7400dd4d7c8a44fb470eb4cbd4" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34315 } }
I20260619 06:49:20.008755 3032 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf73ef7400dd4d7c8a44fb470eb4cbd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf73ef7400dd4d7c8a44fb470eb4cbd4" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34315 } } }
I20260619 06:49:20.009220 3032 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:20.009694 3033 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf73ef7400dd4d7c8a44fb470eb4cbd4. Latest consensus state: current_term: 1 leader_uuid: "bf73ef7400dd4d7c8a44fb470eb4cbd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf73ef7400dd4d7c8a44fb470eb4cbd4" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34315 } } }
I20260619 06:49:20.010011 3033 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:20.014096 3037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:20.019497 3037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:20.028928 3037 catalog_manager.cc:1347] Generated new cluster ID: 7ca59a60ae1e47b0884daca1142d6c1a
I20260619 06:49:20.029059 3037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:20.058521 3037 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:20.059907 3037 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:20.071763 3037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4: Generated new TSK 0
I20260619 06:49:20.072670 3037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:20.079001 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:34315
--builtin_ntp_servers=127.16.79.84:33269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 06:49:20.415258 3050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:20.415539 3050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:20.415702 3050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:20.423246 3050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:20.423350 3050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:20.423523 3050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:49:20.432945 3050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:33269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:34315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:20.434279 3050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:20.436185 3050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:20.449767 3055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:20.449844 3058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:20.449745 3056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:20.450120 3050 server_base.cc:1034] running on GCE node
I20260619 06:49:20.450995 3050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:20.451988 3050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:20.453280 3050 hybrid_clock.cc:648] HybridClock initialized: now 1781851760453221 us; error 45 us; skew 500 ppm
I20260619 06:49:20.453636 3050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:49:20.456038 3050 webserver.cc:460] Webserver started at http://127.16.79.65:41037/ using document root <none> and password file <none>
I20260619 06:49:20.456832 3050 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:20.456970 3050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:20.457327 3050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:20.460426 3050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "1b662e2215014277b9ddfb15084067c9"
format_stamp: "Formatted at 2026-06-19 06:49:20 on dist-test-slave-5w2x"
server_key: "b1f9b97b1d661d496abf485412c79080"
server_key_iv: "3f6e0fc67bcec58a0ade367549c71d76"
server_key_version: "encryptionkey@0"
I20260619 06:49:20.461254 3050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "1b662e2215014277b9ddfb15084067c9"
format_stamp: "Formatted at 2026-06-19 06:49:20 on dist-test-slave-5w2x"
server_key: "b1f9b97b1d661d496abf485412c79080"
server_key_iv: "3f6e0fc67bcec58a0ade367549c71d76"
server_key_version: "encryptionkey@0"
I20260619 06:49:20.466747 3050 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:49:20.470393 3064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:20.471838 3050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:49:20.472034 3050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "1b662e2215014277b9ddfb15084067c9"
format_stamp: "Formatted at 2026-06-19 06:49:20 on dist-test-slave-5w2x"
server_key: "b1f9b97b1d661d496abf485412c79080"
server_key_iv: "3f6e0fc67bcec58a0ade367549c71d76"
server_key_version: "encryptionkey@0"
I20260619 06:49:20.472347 3050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:20.504288 3050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:20.505290 3050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:20.505602 3050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:20.506651 3050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:20.508297 3050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:20.508420 3050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:20.508571 3050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:20.508663 3050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:20.543135 3050 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:39885
I20260619 06:49:20.543223 3176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:39885 every 8 connection(s)
I20260619 06:49:20.544574 3050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:49:20.547413 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 3050
I20260619 06:49:20.547621 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:49:20.548125 16701 external_mini_cluster.cc:1428] Setting key 9bd39351374c37634095627e38edbaaa
I20260619 06:49:20.550928 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:34315
--builtin_ntp_servers=127.16.79.84:33269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 06:49:20.557765 3179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:20.547279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:58861 (local address 127.16.79.126:34315)
0619 06:49:20.547817 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:20.547824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:20.549724 (+ 1900us) server_negotiation.cc:407] Connection header received
0619 06:49:20.551136 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:20.551143 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:20.551231 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:20.551359 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:20.552524 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:20.553263 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:20.554843 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:20.555249 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:20.555377 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:20.556791 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:20.556797 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:20.556802 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:20.556830 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:20.556875 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:20.556887 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:20.556982 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:20.557276 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 06:49:20.557389 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":259,"threads_started":1}
I20260619 06:49:20.558792 3180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:20.547943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:34315 (local address 127.16.79.65:58861)
0619 06:49:20.549036 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:20.549089 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:20.550859 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:20.551606 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:20.551614 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:20.551712 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:20.552361 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:20.552380 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:20.553429 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:20.553434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:20.554683 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:20.554700 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:20.555483 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:20.555489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:20.555722 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:20.556467 (+ 745us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:49:20.556470 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:20.556510 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:49:20.556588 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:20.556981 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:20.556995 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:20.557006 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 06:49:20.557250 (+ 244us) client_negotiation.cc:240] Negotiation successful
0619 06:49:20.557663 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":365,"threads_started":1}
I20260619 06:49:20.560535 3177 heartbeater.cc:347] Connected to a master server at 127.16.79.126:34315
I20260619 06:49:20.560850 3177 heartbeater.cc:464] Registering TS with master...
I20260619 06:49:20.561597 3177 heartbeater.cc:511] Master 127.16.79.126:34315 requested a full tablet report, sending...
I20260619 06:49:20.563953 2991 ts_manager.cc:194] Registered new tserver with Master: 1b662e2215014277b9ddfb15084067c9 (127.16.79.65:39885)
I20260619 06:49:20.566341 2991 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.79.65:58861
W20260619 06:49:20.886065 3181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:20.886349 3181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:20.886435 3181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:20.893957 3181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:20.894068 3181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:20.894177 3181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:49:20.903548 3181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:33269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:34315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:20.904992 3181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:20.906730 3181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:20.920112 3186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:20.920105 3189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:20.920356 3187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:20.921123 3181 server_base.cc:1034] running on GCE node
I20260619 06:49:20.921787 3181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:20.922719 3181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:20.923931 3181 hybrid_clock.cc:648] HybridClock initialized: now 1781851760923887 us; error 30 us; skew 500 ppm
I20260619 06:49:20.924348 3181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:49:20.926784 3181 webserver.cc:460] Webserver started at http://127.16.79.66:42585/ using document root <none> and password file <none>
I20260619 06:49:20.927688 3181 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:20.927824 3181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:20.928295 3181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:20.931208 3181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "c06c3b6e7cf84f97917a98dd1bac2639"
format_stamp: "Formatted at 2026-06-19 06:49:20 on dist-test-slave-5w2x"
server_key: "ce032c534efc0b06fd39c660bf8b43f1"
server_key_iv: "8525b29471ff81e125363827c274e3f0"
server_key_version: "encryptionkey@0"
I20260619 06:49:20.932371 3181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "c06c3b6e7cf84f97917a98dd1bac2639"
format_stamp: "Formatted at 2026-06-19 06:49:20 on dist-test-slave-5w2x"
server_key: "ce032c534efc0b06fd39c660bf8b43f1"
server_key_iv: "8525b29471ff81e125363827c274e3f0"
server_key_version: "encryptionkey@0"
I20260619 06:49:20.937898 3181 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:49:20.941550 3195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:20.943020 3181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:49:20.943171 3181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "c06c3b6e7cf84f97917a98dd1bac2639"
format_stamp: "Formatted at 2026-06-19 06:49:20 on dist-test-slave-5w2x"
server_key: "ce032c534efc0b06fd39c660bf8b43f1"
server_key_iv: "8525b29471ff81e125363827c274e3f0"
server_key_version: "encryptionkey@0"
I20260619 06:49:20.943369 3181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:20.988967 3181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:20.989813 3181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:20.990079 3181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:20.991068 3181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:20.992604 3181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:20.992693 3181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:20.992781 3181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:20.992837 3181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:21.027247 3181 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:41023
I20260619 06:49:21.027335 3307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:41023 every 8 connection(s)
I20260619 06:49:21.028592 3181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:49:21.028890 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 3181
I20260619 06:49:21.029079 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:49:21.029804 16701 external_mini_cluster.cc:1428] Setting key e429067964d6212cd713ec4a95a169db
I20260619 06:49:21.033123 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:34315
--builtin_ntp_servers=127.16.79.84:33269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 06:49:21.048101 3179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.031444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:47769 (local address 127.16.79.126:34315)
0619 06:49:21.031647 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:21.031655 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:21.033478 (+ 1823us) server_negotiation.cc:407] Connection header received
0619 06:49:21.035363 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:21.035371 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:21.035466 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:21.035592 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:21.036845 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.038104 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.039850 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.040585 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.040692 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.047081 (+ 6389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:21.047087 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:21.047090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:21.047116 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:21.047165 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:21.047183 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:21.047270 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:21.047642 (+ 372us) server_negotiation.cc:299] Negotiation successful
0619 06:49:21.047797 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 06:49:21.049934 3310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.032130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:34315 (local address 127.16.79.66:47769)
0619 06:49:21.033317 (+ 1187us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:21.033369 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:21.035064 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:21.035815 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:21.035823 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:21.035919 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:21.036620 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:21.036646 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.038299 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.038303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:21.039683 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:21.039696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.040845 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.040855 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:21.041201 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.046497 (+ 5296us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:49:21.046504 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:21.046576 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:49:21.046764 (+ 188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:21.047321 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:21.047338 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:21.047351 (+ 13us) client_negotiation.cc:769] Sending connection context
0619 06:49:21.047584 (+ 233us) client_negotiation.cc:240] Negotiation successful
0619 06:49:21.048091 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":359,"threads_started":1}
I20260619 06:49:21.051580 3308 heartbeater.cc:347] Connected to a master server at 127.16.79.126:34315
I20260619 06:49:21.051919 3308 heartbeater.cc:464] Registering TS with master...
I20260619 06:49:21.052836 3308 heartbeater.cc:511] Master 127.16.79.126:34315 requested a full tablet report, sending...
I20260619 06:49:21.054487 2991 ts_manager.cc:194] Registered new tserver with Master: c06c3b6e7cf84f97917a98dd1bac2639 (127.16.79.66:41023)
I20260619 06:49:21.055347 2991 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.79.66:47769
W20260619 06:49:21.387382 3311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:21.387741 3311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:21.387912 3311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:21.395493 3311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:21.395663 3311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:21.395856 3311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:49:21.405681 3311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:33269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:34315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:21.407045 3311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:21.408983 3311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:21.422624 3316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:21.422673 3317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:21.422694 3319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:21.423663 3311 server_base.cc:1034] running on GCE node
I20260619 06:49:21.424417 3311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:21.425505 3311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:21.426770 3311 hybrid_clock.cc:648] HybridClock initialized: now 1781851761426725 us; error 42 us; skew 500 ppm
I20260619 06:49:21.427111 3311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:49:21.430166 3311 webserver.cc:460] Webserver started at http://127.16.79.67:42551/ using document root <none> and password file <none>
I20260619 06:49:21.431078 3311 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:21.431226 3311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:21.431564 3311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:21.434777 3311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "cd17ea1e903146d9904e3c0be915e406"
format_stamp: "Formatted at 2026-06-19 06:49:21 on dist-test-slave-5w2x"
server_key: "d426c688ae4db61b61c78eaf9d79e35d"
server_key_iv: "8e5bc14b134be45776218aacb29e802c"
server_key_version: "encryptionkey@0"
I20260619 06:49:21.435639 3311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "cd17ea1e903146d9904e3c0be915e406"
format_stamp: "Formatted at 2026-06-19 06:49:21 on dist-test-slave-5w2x"
server_key: "d426c688ae4db61b61c78eaf9d79e35d"
server_key_iv: "8e5bc14b134be45776218aacb29e802c"
server_key_version: "encryptionkey@0"
I20260619 06:49:21.441160 3311 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260619 06:49:21.444851 3325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:21.446456 3311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:49:21.446677 3311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "cd17ea1e903146d9904e3c0be915e406"
format_stamp: "Formatted at 2026-06-19 06:49:21 on dist-test-slave-5w2x"
server_key: "d426c688ae4db61b61c78eaf9d79e35d"
server_key_iv: "8e5bc14b134be45776218aacb29e802c"
server_key_version: "encryptionkey@0"
I20260619 06:49:21.446918 3311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:21.469308 3311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:21.470233 3311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:21.470562 3311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:21.471664 3311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:21.473335 3311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:21.473460 3311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:21.473601 3311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:21.473697 3311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:21.507797 3311 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:43565
I20260619 06:49:21.507887 3437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:43565 every 8 connection(s)
I20260619 06:49:21.509167 3311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:49:21.512676 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 3311
I20260619 06:49:21.512867 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:49:21.513459 16701 external_mini_cluster.cc:1428] Setting key fe0ceca284679c314beda485b753c977
I20260619 06:49:21.522495 3179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.511818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:57093 (local address 127.16.79.126:34315)
0619 06:49:21.512029 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:21.512035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:21.513881 (+ 1846us) server_negotiation.cc:407] Connection header received
0619 06:49:21.516042 (+ 2161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:21.516047 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:21.516172 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:21.516292 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:21.517360 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.518191 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.519801 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.520280 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.520413 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.521686 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:21.521691 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:21.521695 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:21.521714 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:21.521743 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:21.521755 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:21.521851 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:21.522068 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 06:49:21.522167 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 06:49:21.523545 3440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.512560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:34315 (local address 127.16.79.67:57093)
0619 06:49:21.513651 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:21.513748 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:21.515780 (+ 2032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:21.516528 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:21.516535 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:21.516625 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:21.517193 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:21.517210 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.518336 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.518346 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:21.519657 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:21.519669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.520500 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.520507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:21.520740 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.521356 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:49:21.521359 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:21.521397 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:49:21.521479 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:21.521848 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:21.521860 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:21.521870 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 06:49:21.522059 (+ 189us) client_negotiation.cc:240] Negotiation successful
0619 06:49:21.522499 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":357,"threads_started":1}
I20260619 06:49:21.524983 3438 heartbeater.cc:347] Connected to a master server at 127.16.79.126:34315
I20260619 06:49:21.525377 3438 heartbeater.cc:464] Registering TS with master...
I20260619 06:49:21.526162 3438 heartbeater.cc:511] Master 127.16.79.126:34315 requested a full tablet report, sending...
I20260619 06:49:21.527807 2990 ts_manager.cc:194] Registered new tserver with Master: cd17ea1e903146d9904e3c0be915e406 (127.16.79.67:43565)
I20260619 06:49:21.528982 2990 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.79.67:57093
I20260619 06:49:21.529038 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:49:21.540459 3179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.533511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57190 (local address 127.16.79.126:34315)
0619 06:49:21.533779 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:21.533788 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:21.534177 (+ 389us) server_negotiation.cc:407] Connection header received
0619 06:49:21.534384 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:21.534395 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:21.534465 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:21.534570 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:21.535071 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.535766 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.537589 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.538294 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.538440 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.539650 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:21.539655 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:21.539658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:21.539679 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:21.539710 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:21.539723 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:21.539885 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:21.540084 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 06:49:21.540176 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:49:21.544848 2990 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:49:21.553620 2990 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:49:21.569828 3177 heartbeater.cc:503] Master 127.16.79.126:34315 was elected leader, sending a full tablet report...
I20260619 06:49:21.589795 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.566733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:39885 (local address 127.0.0.1:49762)
0619 06:49:21.580477 (+ 13744us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:21.580500 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:21.580751 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:21.581830 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:21.581836 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:21.581865 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:21.582608 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:21.582626 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.584571 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.584576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:21.586718 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:21.586734 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.589036 (+ 2302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.589042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:21.589203 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.589229 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 06:49:21.589307 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 06:49:21.589414 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13608,"mutex_wait_us":218,"thread_start_us":13469,"threads_started":1}
I20260619 06:49:21.589970 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.566852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49762 (local address 127.16.79.65:39885)
0619 06:49:21.579983 (+ 13131us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:21.579997 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:21.580625 (+ 628us) server_negotiation.cc:407] Connection header received
0619 06:49:21.580947 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:21.580957 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:21.581254 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:21.581437 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:21.582844 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.584366 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.586947 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.588470 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.588613 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.588908 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:21.589383 (+ 475us) server_negotiation.cc:299] Negotiation successful
0619 06:49:21.589566 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12947,"thread_start_us":12802,"threads_started":1}
I20260619 06:49:21.590317 3450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.566702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:43565 (local address 127.0.0.1:45460)
0619 06:49:21.578951 (+ 12249us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:21.579026 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:21.579283 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:21.581760 (+ 2477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:21.581770 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:21.581807 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:21.582609 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:21.582626 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.584469 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.584477 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:21.586778 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:21.586793 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.588710 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.588717 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:21.588960 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.589031 (+ 71us) client_negotiation.cc:769] Sending connection context
0619 06:49:21.589136 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 06:49:21.589340 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11827,"mutex_wait_us":73,"spinlock_wait_cycles":1920,"thread_start_us":11603,"threads_started":1}
I20260619 06:49:21.591616 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.568477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:41023 (local address 127.0.0.1:32984)
0619 06:49:21.582631 (+ 14154us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:21.582647 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:21.582799 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:21.583903 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:21.583909 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:21.583935 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:21.584382 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:21.584396 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.586350 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.586356 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:21.588937 (+ 2581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:21.588949 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.590842 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.590849 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:21.591020 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.591056 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 06:49:21.591148 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 06:49:21.591271 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14036,"mutex_wait_us":1,"thread_start_us":13846,"threads_started":1}
I20260619 06:49:21.591657 3455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.569083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32984 (local address 127.16.79.66:41023)
0619 06:49:21.582013 (+ 12930us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:21.582024 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:21.582766 (+ 742us) server_negotiation.cc:407] Connection header received
0619 06:49:21.582989 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:21.583000 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:21.583267 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:21.583469 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:21.584587 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.586123 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.589149 (+ 3026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.590642 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.590790 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.591008 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:21.591232 (+ 224us) server_negotiation.cc:299] Negotiation successful
0619 06:49:21.591396 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12752,"thread_start_us":12622,"threads_started":1}
I20260619 06:49:21.592725 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.567018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45460 (local address 127.16.79.67:43565)
0619 06:49:21.580934 (+ 13916us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:21.580971 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:21.581003 (+ 32us) server_negotiation.cc:407] Connection header received
0619 06:49:21.581086 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:21.581091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:21.581283 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:21.581443 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:21.582907 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.584302 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.587036 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.588470 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.589810 (+ 1340us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.590906 (+ 1096us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:21.590979 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 06:49:21.591164 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13733,"thread_start_us":13575,"threads_started":1}
I20260619 06:49:21.594403 3112 tablet_service.cc:1467] Processing CreateTablet for tablet 13cb720e89f54e30ad6d70ce9338ddf9 (DEFAULT_TABLE table=test-table [id=58c78b72734a49c4998d4d84209d965b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:21.595546 3243 tablet_service.cc:1467] Processing CreateTablet for tablet 13cb720e89f54e30ad6d70ce9338ddf9 (DEFAULT_TABLE table=test-table [id=58c78b72734a49c4998d4d84209d965b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:21.596092 3112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13cb720e89f54e30ad6d70ce9338ddf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:21.596704 3243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13cb720e89f54e30ad6d70ce9338ddf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:21.596513 3373 tablet_service.cc:1467] Processing CreateTablet for tablet 13cb720e89f54e30ad6d70ce9338ddf9 (DEFAULT_TABLE table=test-table [id=58c78b72734a49c4998d4d84209d965b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:21.597906 3373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13cb720e89f54e30ad6d70ce9338ddf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:21.609596 3456 tablet_bootstrap.cc:492] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639: Bootstrap starting.
I20260619 06:49:21.613049 3458 tablet_bootstrap.cc:492] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406: Bootstrap starting.
I20260619 06:49:21.613175 3456 tablet_bootstrap.cc:654] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:21.614696 3457 tablet_bootstrap.cc:492] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9: Bootstrap starting.
I20260619 06:49:21.616007 3456 log.cc:824] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:21.617605 3458 tablet_bootstrap.cc:654] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:21.618350 3457 tablet_bootstrap.cc:654] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:21.618911 3458 log.cc:824] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:21.619593 3457 log.cc:824] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:21.622210 3456 tablet_bootstrap.cc:492] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639: No bootstrap required, opened a new log
I20260619 06:49:21.622375 3458 tablet_bootstrap.cc:492] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406: No bootstrap required, opened a new log
I20260619 06:49:21.622565 3456 ts_tablet_manager.cc:1397] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260619 06:49:21.622586 3457 tablet_bootstrap.cc:492] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9: No bootstrap required, opened a new log
I20260619 06:49:21.622663 3458 ts_tablet_manager.cc:1397] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260619 06:49:21.622850 3457 ts_tablet_manager.cc:1397] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260619 06:49:21.630363 3456 raft_consensus.cc:348] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b662e2215014277b9ddfb15084067c9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39885 } } peers { permanent_uuid: "cd17ea1e903146d9904e3c0be915e406" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43565 } } peers { permanent_uuid: "c06c3b6e7cf84f97917a98dd1bac2639" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41023 } }
I20260619 06:49:21.630832 3456 raft_consensus.cc:374] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:21.630947 3456 raft_consensus.cc:729] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c06c3b6e7cf84f97917a98dd1bac2639, State: Initialized, Role: FOLLOWER
I20260619 06:49:21.631639 3456 consensus_queue.cc:260] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b662e2215014277b9ddfb15084067c9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39885 } } peers { permanent_uuid: "cd17ea1e903146d9904e3c0be915e406" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43565 } } peers { permanent_uuid: "c06c3b6e7cf84f97917a98dd1bac2639" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41023 } }
I20260619 06:49:21.632859 3308 heartbeater.cc:503] Master 127.16.79.126:34315 was elected leader, sending a full tablet report...
I20260619 06:49:21.633605 3456 ts_tablet_manager.cc:1428] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639: Time spent starting tablet: real 0.011s user 0.000s sys 0.012s
I20260619 06:49:21.633713 3458 raft_consensus.cc:348] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b662e2215014277b9ddfb15084067c9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39885 } } peers { permanent_uuid: "cd17ea1e903146d9904e3c0be915e406" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43565 } } peers { permanent_uuid: "c06c3b6e7cf84f97917a98dd1bac2639" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41023 } }
I20260619 06:49:21.633714 3457 raft_consensus.cc:348] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b662e2215014277b9ddfb15084067c9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39885 } } peers { permanent_uuid: "cd17ea1e903146d9904e3c0be915e406" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43565 } } peers { permanent_uuid: "c06c3b6e7cf84f97917a98dd1bac2639" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41023 } }
I20260619 06:49:21.634160 3457 raft_consensus.cc:374] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:21.634176 3458 raft_consensus.cc:374] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:21.634270 3458 raft_consensus.cc:729] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd17ea1e903146d9904e3c0be915e406, State: Initialized, Role: FOLLOWER
I20260619 06:49:21.634306 3457 raft_consensus.cc:729] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b662e2215014277b9ddfb15084067c9, State: Initialized, Role: FOLLOWER
I20260619 06:49:21.634958 3458 consensus_queue.cc:260] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b662e2215014277b9ddfb15084067c9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39885 } } peers { permanent_uuid: "cd17ea1e903146d9904e3c0be915e406" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43565 } } peers { permanent_uuid: "c06c3b6e7cf84f97917a98dd1bac2639" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41023 } }
I20260619 06:49:21.635012 3457 consensus_queue.cc:260] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b662e2215014277b9ddfb15084067c9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39885 } } peers { permanent_uuid: "cd17ea1e903146d9904e3c0be915e406" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43565 } } peers { permanent_uuid: "c06c3b6e7cf84f97917a98dd1bac2639" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41023 } }
I20260619 06:49:21.637058 3457 ts_tablet_manager.cc:1428] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9: Time spent starting tablet: real 0.014s user 0.008s sys 0.007s
I20260619 06:49:21.637240 3458 ts_tablet_manager.cc:1428] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406: Time spent starting tablet: real 0.014s user 0.007s sys 0.006s
I20260619 06:49:21.637490 3438 heartbeater.cc:503] Master 127.16.79.126:34315 was elected leader, sending a full tablet report...
I20260619 06:49:21.705722 3464 raft_consensus.cc:482] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:49:21.706032 3464 raft_consensus.cc:504] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b662e2215014277b9ddfb15084067c9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39885 } } peers { permanent_uuid: "cd17ea1e903146d9904e3c0be915e406" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43565 } } peers { permanent_uuid: "c06c3b6e7cf84f97917a98dd1bac2639" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41023 } }
I20260619 06:49:21.707779 3464 leader_election.cc:288] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b662e2215014277b9ddfb15084067c9 (127.16.79.65:39885), c06c3b6e7cf84f97917a98dd1bac2639 (127.16.79.66:41023)
I20260619 06:49:21.714401 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.708316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:42069 (local address 127.16.79.65:39885)
0619 06:49:21.708549 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:21.708556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:21.708838 (+ 282us) server_negotiation.cc:407] Connection header received
0619 06:49:21.709093 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:21.709098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:21.709183 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:21.709290 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:21.709739 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.710438 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.712187 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.713298 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.713459 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.713529 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:21.713987 (+ 458us) server_negotiation.cc:299] Negotiation successful
0619 06:49:21.714132 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:49:21.714417 3440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.708385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:39885 (local address 127.16.79.67:42069)
0619 06:49:21.708715 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:21.708733 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:21.708919 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:21.709362 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:21.709367 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:21.709395 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:21.709575 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:21.709583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.710576 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.710584 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:21.711953 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:21.711964 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.713615 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.713620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:21.713793 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.713826 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 06:49:21.713922 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 06:49:21.714041 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":1}
I20260619 06:49:21.715202 3455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.708316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:58215 (local address 127.16.79.66:41023)
0619 06:49:21.708556 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:21.708562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:21.709381 (+ 819us) server_negotiation.cc:407] Connection header received
0619 06:49:21.709650 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:21.709659 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:21.709731 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:21.709834 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:21.710353 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.711044 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.712972 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.714095 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.714265 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.714329 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:21.714737 (+ 408us) server_negotiation.cc:299] Negotiation successful
0619 06:49:21.714878 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 06:49:21.715524 3132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13cb720e89f54e30ad6d70ce9338ddf9" candidate_uuid: "cd17ea1e903146d9904e3c0be915e406" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b662e2215014277b9ddfb15084067c9" is_pre_election: true
I20260619 06:49:21.715847 3465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.708402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:41023 (local address 127.16.79.67:58215)
0619 06:49:21.709260 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:21.709275 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:21.709494 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:21.709846 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:21.709851 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:21.709872 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:21.710206 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:21.710214 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.711213 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.711216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:21.712766 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:21.712782 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.714279 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.714285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:21.714517 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.714557 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 06:49:21.715344 (+ 787us) client_negotiation.cc:240] Negotiation successful
0619 06:49:21.715492 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"mutex_wait_us":52,"spinlock_wait_cycles":30464,"thread_start_us":471,"threads_started":1}
I20260619 06:49:21.716114 3132 raft_consensus.cc:2454] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd17ea1e903146d9904e3c0be915e406 in term 0.
I20260619 06:49:21.716799 3263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13cb720e89f54e30ad6d70ce9338ddf9" candidate_uuid: "cd17ea1e903146d9904e3c0be915e406" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c06c3b6e7cf84f97917a98dd1bac2639" is_pre_election: true
I20260619 06:49:21.716898 3326 leader_election.cc:302] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b662e2215014277b9ddfb15084067c9, cd17ea1e903146d9904e3c0be915e406; no voters:
I20260619 06:49:21.717273 3263 raft_consensus.cc:2454] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd17ea1e903146d9904e3c0be915e406 in term 0.
I20260619 06:49:21.717299 3464 raft_consensus.cc:2781] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:49:21.717463 3464 raft_consensus.cc:482] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:49:21.717558 3464 raft_consensus.cc:3037] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:21.719902 3464 raft_consensus.cc:504] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b662e2215014277b9ddfb15084067c9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39885 } } peers { permanent_uuid: "cd17ea1e903146d9904e3c0be915e406" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43565 } } peers { permanent_uuid: "c06c3b6e7cf84f97917a98dd1bac2639" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41023 } }
I20260619 06:49:21.720675 3464 leader_election.cc:288] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [CANDIDATE]: Term 1 election: Requested vote from peers 1b662e2215014277b9ddfb15084067c9 (127.16.79.65:39885), c06c3b6e7cf84f97917a98dd1bac2639 (127.16.79.66:41023)
I20260619 06:49:21.721253 3132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13cb720e89f54e30ad6d70ce9338ddf9" candidate_uuid: "cd17ea1e903146d9904e3c0be915e406" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b662e2215014277b9ddfb15084067c9"
I20260619 06:49:21.721340 3263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13cb720e89f54e30ad6d70ce9338ddf9" candidate_uuid: "cd17ea1e903146d9904e3c0be915e406" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c06c3b6e7cf84f97917a98dd1bac2639"
I20260619 06:49:21.721487 3132 raft_consensus.cc:3037] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:21.721616 3263 raft_consensus.cc:3037] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:21.723871 3132 raft_consensus.cc:2454] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd17ea1e903146d9904e3c0be915e406 in term 1.
I20260619 06:49:21.723908 3263 raft_consensus.cc:2454] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd17ea1e903146d9904e3c0be915e406 in term 1.
I20260619 06:49:21.724421 3326 leader_election.cc:302] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b662e2215014277b9ddfb15084067c9, cd17ea1e903146d9904e3c0be915e406; no voters:
I20260619 06:49:21.724751 3464 raft_consensus.cc:2781] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:21.725262 3464 raft_consensus.cc:686] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [term 1 LEADER]: Becoming Leader. State: Replica: cd17ea1e903146d9904e3c0be915e406, State: Running, Role: LEADER
I20260619 06:49:21.725777 3464 consensus_queue.cc:237] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b662e2215014277b9ddfb15084067c9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39885 } } peers { permanent_uuid: "cd17ea1e903146d9904e3c0be915e406" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43565 } } peers { permanent_uuid: "c06c3b6e7cf84f97917a98dd1bac2639" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41023 } }
I20260619 06:49:21.730786 2989 catalog_manager.cc:5495] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd17ea1e903146d9904e3c0be915e406 (127.16.79.67). New cstate: current_term: 1 leader_uuid: "cd17ea1e903146d9904e3c0be915e406" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b662e2215014277b9ddfb15084067c9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd17ea1e903146d9904e3c0be915e406" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 43565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c06c3b6e7cf84f97917a98dd1bac2639" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41023 } health_report { overall_health: UNKNOWN } } }
I20260619 06:49:21.757222 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.751467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45466 (local address 127.16.79.67:43565)
0619 06:49:21.751677 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:21.751681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:21.751764 (+ 83us) server_negotiation.cc:407] Connection header received
0619 06:49:21.752016 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:21.752021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:21.752122 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:21.752312 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:49:21.752726 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.753465 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.754901 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.755323 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.755416 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.755989 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:49:21.756602 (+ 613us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:49:21.756755 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:21.756978 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 06:49:21.757019 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260619 06:49:21.760914 3439 tablet.cc:2369] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:49:21.767051 3263 raft_consensus.cc:1261] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639 [term 1 FOLLOWER]: Refusing update from remote peer cd17ea1e903146d9904e3c0be915e406: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:49:21.767055 3132 raft_consensus.cc:1261] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9 [term 1 FOLLOWER]: Refusing update from remote peer cd17ea1e903146d9904e3c0be915e406: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:49:21.768213 3466 consensus_queue.cc:1035] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b662e2215014277b9ddfb15084067c9" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 39885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:49:21.768601 3464 consensus_queue.cc:1035] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06c3b6e7cf84f97917a98dd1bac2639" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 41023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 06:49:21.781057 3309 tablet.cc:2369] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:49:21.800769 2989 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:49:21.801093 2989 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:49:21.805449 2989 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf73ef7400dd4d7c8a44fb470eb4cbd4: Sending DeleteTablet for 3 replicas of tablet 13cb720e89f54e30ad6d70ce9338ddf9
I20260619 06:49:21.806785 3373 tablet_service.cc:1514] Processing DeleteTablet for tablet 13cb720e89f54e30ad6d70ce9338ddf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:21 UTC) from {username='slave'} at 127.0.0.1:45460
I20260619 06:49:21.806906 3112 tablet_service.cc:1514] Processing DeleteTablet for tablet 13cb720e89f54e30ad6d70ce9338ddf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:21 UTC) from {username='slave'} at 127.0.0.1:49762
I20260619 06:49:21.807108 3243 tablet_service.cc:1514] Processing DeleteTablet for tablet 13cb720e89f54e30ad6d70ce9338ddf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:21 UTC) from {username='slave'} at 127.0.0.1:32984
I20260619 06:49:21.809329 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 3050
I20260619 06:49:21.818881 3481 tablet_replica.cc:330] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639: stopping tablet replica
I20260619 06:49:21.819689 3479 tablet_replica.cc:330] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9: stopping tablet replica
I20260619 06:49:21.820150 3479 raft_consensus.cc:2229] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:49:21.820731 3479 raft_consensus.cc:2258] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:21.819379 3481 raft_consensus.cc:2229] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:49:21.821502 3481 raft_consensus.cc:2258] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:21.823856 3479 ts_tablet_manager.cc:1905] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:21.825649 3481 ts_tablet_manager.cc:1905] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:21.828735 3480 tablet_replica.cc:330] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406: stopping tablet replica
I20260619 06:49:21.829210 3480 raft_consensus.cc:2229] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:49:21.830013 3480 raft_consensus.cc:2258] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:21.832664 3480 ts_tablet_manager.cc:1905] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:21.833683 3479 ts_tablet_manager.cc:1918] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:21.833894 3479 log.cc:1196] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/13cb720e89f54e30ad6d70ce9338ddf9
I20260619 06:49:21.834636 3479 ts_tablet_manager.cc:1939] T 13cb720e89f54e30ad6d70ce9338ddf9 P 1b662e2215014277b9ddfb15084067c9: Deleting consensus metadata
I20260619 06:49:21.836759 2975 catalog_manager.cc:4841] TS 1b662e2215014277b9ddfb15084067c9 (127.16.79.65:39885): tablet 13cb720e89f54e30ad6d70ce9338ddf9 (table test-table [id=58c78b72734a49c4998d4d84209d965b]) successfully deleted
I20260619 06:49:21.839540 3481 ts_tablet_manager.cc:1918] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:21.839748 3481 log.cc:1196] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/13cb720e89f54e30ad6d70ce9338ddf9
I20260619 06:49:21.840339 3481 ts_tablet_manager.cc:1939] T 13cb720e89f54e30ad6d70ce9338ddf9 P c06c3b6e7cf84f97917a98dd1bac2639: Deleting consensus metadata
I20260619 06:49:21.841074 3480 ts_tablet_manager.cc:1918] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:21.841308 3480 log.cc:1196] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/13cb720e89f54e30ad6d70ce9338ddf9
I20260619 06:49:21.841955 3480 ts_tablet_manager.cc:1939] T 13cb720e89f54e30ad6d70ce9338ddf9 P cd17ea1e903146d9904e3c0be915e406: Deleting consensus metadata
I20260619 06:49:21.842048 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.811089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49774 (local address 127.16.79.65:39885)
0619 06:49:21.811291 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:21.811298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:21.833269 (+ 21971us) server_negotiation.cc:407] Connection header received
0619 06:49:21.833808 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:21.833815 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:21.833925 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:21.834006 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:21.834796 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.835936 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.837948 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.838580 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.838725 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.840243 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:21.840255 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:21.840275 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:21.841085 (+ 810us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:21.841148 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:21.841180 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:21.841275 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:21.841539 (+ 264us) server_negotiation.cc:299] Negotiation successful
0619 06:49:21.841662 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 06:49:21.842367 2978 catalog_manager.cc:4841] TS c06c3b6e7cf84f97917a98dd1bac2639 (127.16.79.66:41023): tablet 13cb720e89f54e30ad6d70ce9338ddf9 (table test-table [id=58c78b72734a49c4998d4d84209d965b]) successfully deleted
I20260619 06:49:21.843418 3172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:21.844198 2976 catalog_manager.cc:4841] TS cd17ea1e903146d9904e3c0be915e406 (127.16.79.67:43565): tablet 13cb720e89f54e30ad6d70ce9338ddf9 (table test-table [id=58c78b72734a49c4998d4d84209d965b]) successfully deleted
I20260619 06:49:21.937351 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 3050
I20260619 06:49:21.955734 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 3181
I20260619 06:49:21.964002 3455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:21.956736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32996 (local address 127.16.79.66:41023)
0619 06:49:21.956921 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:21.956926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:21.956945 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:49:21.957079 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:21.957083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:21.957198 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:21.957286 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:21.957749 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.958567 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.960588 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:21.960999 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:21.961095 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:21.962396 (+ 1301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:21.962407 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:21.962427 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:21.963189 (+ 762us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:21.963255 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:21.963281 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:21.963389 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:21.963541 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 06:49:21.963656 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:49:21.964905 3303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:22.062402 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 3181
I20260619 06:49:22.080102 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 3311
I20260619 06:49:22.088363 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:22.080969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45468 (local address 127.16.79.67:43565)
0619 06:49:22.081196 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:22.081201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:22.081226 (+ 25us) server_negotiation.cc:407] Connection header received
0619 06:49:22.081355 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:22.081361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:22.081469 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:22.081588 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:22.081964 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:22.083184 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:22.084530 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:22.085236 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:22.085369 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:22.086420 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:22.086433 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:22.086462 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:22.087304 (+ 842us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:22.087381 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:22.087407 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:22.087533 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:22.087673 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 06:49:22.087783 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 06:49:22.089068 3433 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:22.211404 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 3311
I20260619 06:49:22.231040 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 2960
I20260619 06:49:22.232196 3021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:22.346681 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 2960
2026-06-19T06:49:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2900 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:22 dist-test-slave-5w2x krb5kdc[3489](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:22 dist-test-slave-5w2x krb5kdc[3489](info): set up 2 sockets
Jun 19 06:49:22 dist-test-slave-5w2x krb5kdc[3489](info): commencing operation
krb5kdc: starting...
W20260619 06:49:23.540691 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.007s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:23 dist-test-slave-5w2x krb5kdc[3489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:49:23.797772 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:42935
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:42935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:24.134640 3502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:24.134974 3502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:24.135149 3502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:24.142685 3502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:24.142813 3502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:24.142923 3502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:24.143011 3502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:24.143080 3502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:24.153326 3502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:42935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:42935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:24.154757 3502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:24.156186 3502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:24.168673 3507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:24.168789 3508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:24.168766 3510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:24.169319 3502 server_base.cc:1034] running on GCE node
I20260619 06:49:24.170260 3502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:24.171775 3502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:24.173008 3502 hybrid_clock.cc:648] HybridClock initialized: now 1781851764172966 us; error 41 us; skew 500 ppm
I20260619 06:49:24.173434 3502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:24 dist-test-slave-5w2x krb5kdc[3489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851764, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:24.179795 3502 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:49:24.181533 3502 webserver.cc:460] Webserver started at http://127.16.79.126:40179/ using document root <none> and password file <none>
I20260619 06:49:24.182261 3502 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:24.182411 3502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:24.182730 3502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:24.185912 3502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "d6858f56424b48d1befcd537d75c1a43"
format_stamp: "Formatted at 2026-06-19 06:49:24 on dist-test-slave-5w2x"
server_key: "459539bcd666f837023c58246407297b"
server_key_iv: "9402b653de666e926192af78c434abf2"
server_key_version: "encryptionkey@0"
I20260619 06:49:24.186691 3502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "d6858f56424b48d1befcd537d75c1a43"
format_stamp: "Formatted at 2026-06-19 06:49:24 on dist-test-slave-5w2x"
server_key: "459539bcd666f837023c58246407297b"
server_key_iv: "9402b653de666e926192af78c434abf2"
server_key_version: "encryptionkey@0"
I20260619 06:49:24.192101 3502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:49:24.195703 3517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:24.197082 3502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:49:24.197268 3502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "d6858f56424b48d1befcd537d75c1a43"
format_stamp: "Formatted at 2026-06-19 06:49:24 on dist-test-slave-5w2x"
server_key: "459539bcd666f837023c58246407297b"
server_key_iv: "9402b653de666e926192af78c434abf2"
server_key_version: "encryptionkey@0"
I20260619 06:49:24.197471 3502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:24.214986 3502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:24.219405 3502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:24.219733 3502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:24.239504 3502 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:42935
I20260619 06:49:24.239540 3568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:42935 every 8 connection(s)
I20260619 06:49:24.241112 3502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:24.242964 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 3502
I20260619 06:49:24.243184 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:24.243734 16701 external_mini_cluster.cc:1428] Setting key 6fbf1396fc4cd21d2816720e4e2d0351
I20260619 06:49:24.246212 3569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 06:49:24 dist-test-slave-5w2x krb5kdc[3489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:49:24.259308 3569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43: Bootstrap starting.
I20260619 06:49:24.263154 3569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:24.264472 3569 log.cc:824] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:24.267910 3569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43: No bootstrap required, opened a new log
I20260619 06:49:24.268415 3572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:24.245731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36200 (local address 127.16.79.126:42935)
0619 06:49:24.246564 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:24.246576 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:24.246619 (+ 43us) server_negotiation.cc:407] Connection header received
0619 06:49:24.248352 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:24.248375 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:24.248800 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:24.249154 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:24.251031 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:24.252582 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:24.254403 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:24.255088 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:24.255248 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:24.260298 (+ 5050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:24.260313 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:24.260329 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:24.260371 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:24.263750 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:24.264774 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:24.264781 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:24.264787 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:24.264997 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:24.265492 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:24.265500 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:24.265506 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:24.266181 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:24.266404 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:24.266870 (+ 466us) server_negotiation.cc:299] Negotiation successful
0619 06:49:24.267204 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":390,"threads_started":1}
I20260619 06:49:24.277205 3569 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6858f56424b48d1befcd537d75c1a43" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42935 } }
I20260619 06:49:24.277670 3569 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:24.277784 3569 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6858f56424b48d1befcd537d75c1a43, State: Initialized, Role: FOLLOWER
I20260619 06:49:24.278409 3569 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6858f56424b48d1befcd537d75c1a43" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42935 } }
I20260619 06:49:24.278668 3569 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:24.278817 3569 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:24.278992 3569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:24.281268 3569 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6858f56424b48d1befcd537d75c1a43" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42935 } }
I20260619 06:49:24.281813 3569 leader_election.cc:302] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6858f56424b48d1befcd537d75c1a43; no voters:
I20260619 06:49:24.282419 3569 leader_election.cc:288] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:24.282593 3574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:24.283409 3574 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [term 1 LEADER]: Becoming Leader. State: Replica: d6858f56424b48d1befcd537d75c1a43, State: Running, Role: LEADER
I20260619 06:49:24.284013 3574 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6858f56424b48d1befcd537d75c1a43" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42935 } }
I20260619 06:49:24.284847 3569 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:24.287087 3576 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6858f56424b48d1befcd537d75c1a43. Latest consensus state: current_term: 1 leader_uuid: "d6858f56424b48d1befcd537d75c1a43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6858f56424b48d1befcd537d75c1a43" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42935 } } }
I20260619 06:49:24.287050 3575 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6858f56424b48d1befcd537d75c1a43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6858f56424b48d1befcd537d75c1a43" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 42935 } } }
I20260619 06:49:24.287654 3575 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:24.287524 3576 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:24.289202 3579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:24.293928 3579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:24.303999 3579 catalog_manager.cc:1347] Generated new cluster ID: 696bc5cb66524441b10b5ef502ec3482
I20260619 06:49:24.304163 3579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:24.328940 3579 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:24.330375 3579 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:24.343210 3579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43: Generated new TSK 0
I20260619 06:49:24.344276 3579 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.16.79.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
Entry for principal kudu/127.16.79.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab.
I20260619 06:49:24.404050 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42935
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:24.759048 3595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:24.759382 3595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:24.759554 3595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:24.767436 3595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:24.767620 3595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:24.767807 3595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:49:24.778360 3595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.65.keytab
--principal=kudu/127.16.79.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:24.779798 3595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:24.781224 3595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:24.795737 3600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:24.795558 3603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:24.795599 3601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:24.796589 3595 server_base.cc:1034] running on GCE node
I20260619 06:49:24.797456 3595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:24.798470 3595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:24.799696 3595 hybrid_clock.cc:648] HybridClock initialized: now 1781851764799659 us; error 40 us; skew 500 ppm
I20260619 06:49:24.800037 3595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:24 dist-test-slave-5w2x krb5kdc[3489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851764, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:24.803882 3595 init.cc:377] Logged in from keytab as kudu/127.16.79.65@KRBTEST.COM (short username kudu)
I20260619 06:49:24.805784 3595 webserver.cc:460] Webserver started at http://127.16.79.65:38629/ using document root <none> and password file <none>
I20260619 06:49:24.806551 3595 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:24.806721 3595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:24.807066 3595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:24.810278 3595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "a7b67305ab7149a1ad955d29c79e3ec8"
format_stamp: "Formatted at 2026-06-19 06:49:24 on dist-test-slave-5w2x"
server_key: "9d3efd584614d1ca5269350cd31a4de1"
server_key_iv: "0861358cea176dc1b9dd4388da9be78c"
server_key_version: "encryptionkey@0"
I20260619 06:49:24.811131 3595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "a7b67305ab7149a1ad955d29c79e3ec8"
format_stamp: "Formatted at 2026-06-19 06:49:24 on dist-test-slave-5w2x"
server_key: "9d3efd584614d1ca5269350cd31a4de1"
server_key_iv: "0861358cea176dc1b9dd4388da9be78c"
server_key_version: "encryptionkey@0"
I20260619 06:49:24.816972 3595 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 06:49:24.821020 3610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:24.822532 3595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:49:24.822751 3595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "a7b67305ab7149a1ad955d29c79e3ec8"
format_stamp: "Formatted at 2026-06-19 06:49:24 on dist-test-slave-5w2x"
server_key: "9d3efd584614d1ca5269350cd31a4de1"
server_key_iv: "0861358cea176dc1b9dd4388da9be78c"
server_key_version: "encryptionkey@0"
I20260619 06:49:24.822991 3595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:24.843634 3595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:24.848441 3595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:24.848747 3595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:24.849777 3595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:24.851212 3595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:24.851322 3595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:24.851476 3595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:24.851548 3595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:24.886981 3595 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:36473
I20260619 06:49:24.887033 3722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:36473 every 8 connection(s)
I20260619 06:49:24.888350 3595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:49:24.893728 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 3595
I20260619 06:49:24.893921 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:49:24.894461 16701 external_mini_cluster.cc:1428] Setting key b714d7726c3efbe078431f26f93067cb
Jun 19 06:49:24 dist-test-slave-5w2x krb5kdc[3489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851764, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.65@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:49:24.914307 3725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:24.891032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:36869 (local address 127.16.79.126:42935)
0619 06:49:24.891538 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:24.891542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:24.892451 (+ 909us) server_negotiation.cc:407] Connection header received
0619 06:49:24.895091 (+ 2640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:24.895097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:24.895201 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:24.895350 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:24.897775 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:24.898839 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:24.901630 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:24.902326 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:24.902460 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:24.907612 (+ 5152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:24.907618 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:24.907621 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:24.907660 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:24.910339 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:24.911499 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:24.911504 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:24.911506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:24.911665 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:24.912294 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:24.912298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:24.912300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:24.912604 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:24.912765 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:24.913802 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0619 06:49:24.914021 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":293,"threads_started":1}
I20260619 06:49:24.916049 3726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:24.891427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:42935 (local address 127.16.79.65:36869)
0619 06:49:24.892258 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:24.892306 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:24.894708 (+ 2402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:24.895722 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:24.895732 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:24.896573 (+ 841us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:24.897541 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:24.897577 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:24.899044 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:24.899060 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:24.901422 (+ 2362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:24.901446 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:24.902621 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:24.902631 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:24.903009 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:24.903908 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:49:24.903951 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:24.907263 (+ 3312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:24.910530 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:24.910543 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:24.910566 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:24.911314 (+ 748us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:24.911809 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:24.911816 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:24.911823 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:24.912109 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:24.912754 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:24.912764 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:24.913325 (+ 561us) client_negotiation.cc:769] Sending connection context
0619 06:49:24.913666 (+ 341us) client_negotiation.cc:240] Negotiation successful
0619 06:49:24.914333 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":295,"threads_started":1}
I20260619 06:49:24.918015 3723 heartbeater.cc:347] Connected to a master server at 127.16.79.126:42935
I20260619 06:49:24.918414 3723 heartbeater.cc:464] Registering TS with master...
I20260619 06:49:24.919315 3723 heartbeater.cc:511] Master 127.16.79.126:42935 requested a full tablet report, sending...
I20260619 06:49:24.921804 3534 ts_manager.cc:194] Registered new tserver with Master: a7b67305ab7149a1ad955d29c79e3ec8 (127.16.79.65:36473)
WARNING: no policy specified for kudu/127.16.79.66@KRBTEST.COM; defaulting to no policy
I20260619 06:49:24.923388 3534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.65@KRBTEST.COM'} at 127.16.79.65:36869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
Entry for principal kudu/127.16.79.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab.
I20260619 06:49:24.955981 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42935
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:25.302649 3729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:25.302947 3729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:25.303133 3729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:25.310629 3729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:25.310743 3729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:25.310940 3729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:49:25.321596 3729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.66.keytab
--principal=kudu/127.16.79.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:25.322963 3729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:25.324409 3729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:25.338492 3737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:25.338673 3735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:25.338608 3734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:25.339704 3729 server_base.cc:1034] running on GCE node
I20260619 06:49:25.340570 3729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:25.341562 3729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:25.342787 3729 hybrid_clock.cc:648] HybridClock initialized: now 1781851765342713 us; error 64 us; skew 500 ppm
I20260619 06:49:25.343184 3729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:25 dist-test-slave-5w2x krb5kdc[3489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851765, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:25.346498 3729 init.cc:377] Logged in from keytab as kudu/127.16.79.66@KRBTEST.COM (short username kudu)
I20260619 06:49:25.348085 3729 webserver.cc:460] Webserver started at http://127.16.79.66:46179/ using document root <none> and password file <none>
I20260619 06:49:25.348819 3729 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:25.348964 3729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:25.349287 3729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:25.352397 3729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "21a48b141ddc45448ac66edd5df96094"
format_stamp: "Formatted at 2026-06-19 06:49:25 on dist-test-slave-5w2x"
server_key: "31f13a4d38d2d6228c9833382826cee3"
server_key_iv: "cb4b0ffb4b4ac36c5c3079306f29f067"
server_key_version: "encryptionkey@0"
I20260619 06:49:25.353219 3729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "21a48b141ddc45448ac66edd5df96094"
format_stamp: "Formatted at 2026-06-19 06:49:25 on dist-test-slave-5w2x"
server_key: "31f13a4d38d2d6228c9833382826cee3"
server_key_iv: "cb4b0ffb4b4ac36c5c3079306f29f067"
server_key_version: "encryptionkey@0"
I20260619 06:49:25.358790 3729 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 06:49:25.362365 3744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:25.363754 3729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:49:25.363955 3729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "21a48b141ddc45448ac66edd5df96094"
format_stamp: "Formatted at 2026-06-19 06:49:25 on dist-test-slave-5w2x"
server_key: "31f13a4d38d2d6228c9833382826cee3"
server_key_iv: "cb4b0ffb4b4ac36c5c3079306f29f067"
server_key_version: "encryptionkey@0"
I20260619 06:49:25.364249 3729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:25.386657 3729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:25.391230 3729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:25.391534 3729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:25.392551 3729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:25.393932 3729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:25.394044 3729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:25.394202 3729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:25.394277 3729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:25.428421 3729 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:38019
I20260619 06:49:25.428506 3856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:38019 every 8 connection(s)
I20260619 06:49:25.429759 3729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:49:25.431854 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 3729
I20260619 06:49:25.432034 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:49:25.432596 16701 external_mini_cluster.cc:1428] Setting key 1bdb106712f8fc08a6b21912020ce4c9
Jun 19 06:49:25 dist-test-slave-5w2x krb5kdc[3489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851765, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.66@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:49:25.452986 3859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:25.432569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:52937 (local address 127.16.79.126:42935)
0619 06:49:25.436550 (+ 3981us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:25.436558 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:25.436584 (+ 26us) server_negotiation.cc:407] Connection header received
0619 06:49:25.437171 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:25.437180 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:25.437280 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:25.437398 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:25.438952 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:25.440192 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:25.441764 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:25.442531 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:25.442692 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:25.447047 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:25.447052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:25.447055 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:25.447087 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:25.449778 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:25.450530 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:25.450536 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:25.450540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:25.450681 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:25.451114 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:25.451120 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:25.451123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:25.451396 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:25.451571 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:25.452355 (+ 784us) server_negotiation.cc:299] Negotiation successful
0619 06:49:25.452603 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3830,"thread_start_us":3717,"threads_started":1}
I20260619 06:49:25.454075 3860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:25.433083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:42935 (local address 127.16.79.66:52937)
0619 06:49:25.434099 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:25.434169 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:25.436884 (+ 2715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:25.437618 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:25.437629 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:25.438192 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:25.438761 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:25.438779 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:25.440326 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:25.440330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:25.441582 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:25.441594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:25.442750 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:25.442755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:25.443086 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:25.444015 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:49:25.444050 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:25.446809 (+ 2759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:25.449933 (+ 3124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:25.449942 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:25.449961 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:25.450403 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:25.450792 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:25.450797 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:25.450801 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:25.450997 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:25.451543 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:25.451552 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:25.451930 (+ 378us) client_negotiation.cc:769] Sending connection context
0619 06:49:25.452261 (+ 331us) client_negotiation.cc:240] Negotiation successful
0619 06:49:25.452840 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":376,"threads_started":1}
I20260619 06:49:25.455690 3857 heartbeater.cc:347] Connected to a master server at 127.16.79.126:42935
I20260619 06:49:25.456053 3857 heartbeater.cc:464] Registering TS with master...
I20260619 06:49:25.456976 3857 heartbeater.cc:511] Master 127.16.79.126:42935 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.16.79.67@KRBTEST.COM; defaulting to no policy
I20260619 06:49:25.458722 3534 ts_manager.cc:194] Registered new tserver with Master: 21a48b141ddc45448ac66edd5df96094 (127.16.79.66:38019)
I20260619 06:49:25.459651 3534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.66@KRBTEST.COM'} at 127.16.79.66:52937
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
Entry for principal kudu/127.16.79.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab.
I20260619 06:49:25.489774 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42935
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:44951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:25.825183 3863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:25.825485 3863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:25.825654 3863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:25.833405 3863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:25.833518 3863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:25.833711 3863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:49:25.843678 3863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:44951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.67.keytab
--principal=kudu/127.16.79.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:42935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:25.845320 3863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:25.846695 3863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:25.860512 3871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:25.860620 3868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:25.860596 3869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:25.861495 3863 server_base.cc:1034] running on GCE node
I20260619 06:49:25.862169 3863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:25.863159 3863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:25.864413 3863 hybrid_clock.cc:648] HybridClock initialized: now 1781851765864343 us; error 61 us; skew 500 ppm
I20260619 06:49:25.864765 3863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:25 dist-test-slave-5w2x krb5kdc[3489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851765, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:25.868075 3863 init.cc:377] Logged in from keytab as kudu/127.16.79.67@KRBTEST.COM (short username kudu)
I20260619 06:49:25.869712 3863 webserver.cc:460] Webserver started at http://127.16.79.67:46227/ using document root <none> and password file <none>
I20260619 06:49:25.870393 3863 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:25.870496 3863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:25.870766 3863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:25.873795 3863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "b881e5df4a8645129f94d46ef0d57eab"
format_stamp: "Formatted at 2026-06-19 06:49:25 on dist-test-slave-5w2x"
server_key: "648673284016e402acd9915b896cd97e"
server_key_iv: "d825488ccdc2a2d90601c1713f8cdc63"
server_key_version: "encryptionkey@0"
I20260619 06:49:25.874529 3863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "b881e5df4a8645129f94d46ef0d57eab"
format_stamp: "Formatted at 2026-06-19 06:49:25 on dist-test-slave-5w2x"
server_key: "648673284016e402acd9915b896cd97e"
server_key_iv: "d825488ccdc2a2d90601c1713f8cdc63"
server_key_version: "encryptionkey@0"
I20260619 06:49:25.880105 3863 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:49:25.883999 3878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:25.885596 3863 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260619 06:49:25.885775 3863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "b881e5df4a8645129f94d46ef0d57eab"
format_stamp: "Formatted at 2026-06-19 06:49:25 on dist-test-slave-5w2x"
server_key: "648673284016e402acd9915b896cd97e"
server_key_iv: "d825488ccdc2a2d90601c1713f8cdc63"
server_key_version: "encryptionkey@0"
I20260619 06:49:25.885986 3863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:25.905501 3863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:25.909655 3863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:25.909906 3863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:25.910861 3863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:25.912266 3863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:25.912344 3863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:25.912433 3863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:25.912477 3863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:25.927166 3723 heartbeater.cc:503] Master 127.16.79.126:42935 was elected leader, sending a full tablet report...
I20260619 06:49:25.947793 3863 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:39071
I20260619 06:49:25.947840 3990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:39071 every 8 connection(s)
I20260619 06:49:25.949189 3863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:49:25.955675 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 3863
I20260619 06:49:25.955862 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:49:25.956418 16701 external_mini_cluster.cc:1428] Setting key 4eac59026a3cce2886f3bb71a346f354
Jun 19 06:49:25 dist-test-slave-5w2x krb5kdc[3489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851765, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.67@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:49:25.969751 3859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:25.951785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:33873 (local address 127.16.79.126:42935)
0619 06:49:25.951971 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:25.951975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:25.953176 (+ 1201us) server_negotiation.cc:407] Connection header received
0619 06:49:25.955035 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:25.955041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:25.955139 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:25.955274 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:25.957137 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:25.958253 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:25.960057 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:25.960514 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:25.960645 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:25.964048 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:25.964054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:25.964057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:25.964093 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:25.966884 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:25.967598 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:25.967602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:25.967605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:25.967731 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:25.968178 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:25.968188 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:25.968191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:25.968511 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:25.968628 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:25.969284 (+ 656us) server_negotiation.cc:299] Negotiation successful
0619 06:49:25.969458 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 06:49:25.971016 3993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:25.952290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:42935 (local address 127.16.79.67:33873)
0619 06:49:25.953005 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:25.953043 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:25.954724 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:25.955519 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:25.955528 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:25.956214 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:25.956894 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:25.956920 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:25.958441 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:25.958445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:25.959918 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:25.959931 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:25.960725 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:25.960741 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:25.960949 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:25.961576 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:49:25.961602 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:25.963840 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:25.967032 (+ 3192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:25.967039 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:25.967051 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:25.967490 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:25.967825 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:49:25.967829 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:49:25.967831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:49:25.968009 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:49:25.968619 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:25.968625 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:25.968963 (+ 338us) client_negotiation.cc:769] Sending connection context
0619 06:49:25.969197 (+ 234us) client_negotiation.cc:240] Negotiation successful
0619 06:49:25.969608 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":267,"threads_started":1}
I20260619 06:49:25.972460 3991 heartbeater.cc:347] Connected to a master server at 127.16.79.126:42935
I20260619 06:49:25.972776 3991 heartbeater.cc:464] Registering TS with master...
I20260619 06:49:25.973502 3991 heartbeater.cc:511] Master 127.16.79.126:42935 requested a full tablet report, sending...
I20260619 06:49:25.974900 3534 ts_manager.cc:194] Registered new tserver with Master: b881e5df4a8645129f94d46ef0d57eab (127.16.79.67:39071)
I20260619 06:49:25.975883 3534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.16.79.67@KRBTEST.COM'} at 127.16.79.67:33873
I20260619 06:49:25.984874 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:49:26.002777 3859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:25.989446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33732 (local address 127.16.79.126:42935)
0619 06:49:25.989718 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:25.989727 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:25.990027 (+ 300us) server_negotiation.cc:407] Connection header received
0619 06:49:25.990353 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:25.990357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:25.990425 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:25.990513 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:25.991932 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:25.992717 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:25.994025 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:25.994460 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:25.994556 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:25.996932 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:25.996983 (+ 51us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:25.996995 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:25.997038 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:25.999773 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:26.000613 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:26.000617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:26.000620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:26.000832 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:26.001231 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:26.001235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:26.001237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:26.001598 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:26.001860 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:26.002194 (+ 334us) server_negotiation.cc:299] Negotiation successful
0619 06:49:26.002372 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260619 06:49:26.007004 3534 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:49:26.013995 3534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:49:26.040280 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:26.026875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37140 (local address 127.16.79.65:36473)
0619 06:49:26.027732 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:26.027741 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:26.028297 (+ 556us) server_negotiation.cc:407] Connection header received
0619 06:49:26.028875 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:26.028887 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:26.029212 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:26.029405 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:26.030631 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.032812 (+ 2181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.036740 (+ 3928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.038901 (+ 2161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.039026 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:26.039291 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:26.039749 (+ 458us) server_negotiation.cc:299] Negotiation successful
0619 06:49:26.039920 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":437,"threads_started":1}
I20260619 06:49:26.040637 4003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:26.026546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:36473 (local address 127.0.0.1:37140)
0619 06:49:26.028119 (+ 1573us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:26.028187 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:26.028467 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:26.029832 (+ 1365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:26.029843 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:26.029885 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:26.030369 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:26.030385 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.033064 (+ 2679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.033073 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:26.036358 (+ 3285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:26.036381 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.039277 (+ 2896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.039284 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:26.039511 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:26.039551 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 06:49:26.039697 (+ 146us) client_negotiation.cc:240] Negotiation successful
0619 06:49:26.039902 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1311,"thread_start_us":1169,"threads_started":1}
I20260619 06:49:26.045325 3658 tablet_service.cc:1467] Processing CreateTablet for tablet 289611f3165b4033ba218ad191a8780c (DEFAULT_TABLE table=test-table [id=5f56b27c3ddb40a8802a88acc7b0ca81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:26.046981 3658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 289611f3165b4033ba218ad191a8780c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:26.047367 4008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:26.029167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47368 (local address 127.16.79.67:39071)
0619 06:49:26.038440 (+ 9273us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:26.038451 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:26.038489 (+ 38us) server_negotiation.cc:407] Connection header received
0619 06:49:26.039480 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:26.039487 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:26.039671 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:26.039852 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:26.040930 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.042396 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.044530 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.045951 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.046084 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:26.046319 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:26.046889 (+ 570us) server_negotiation.cc:299] Negotiation successful
0619 06:49:26.047047 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9079,"thread_start_us":9339,"threads_started":1}
I20260619 06:49:26.047427 4007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:26.028596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:39071 (local address 127.0.0.1:47368)
0619 06:49:26.038078 (+ 9482us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:26.038102 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:26.038235 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:26.040177 (+ 1942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:26.040182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:26.040203 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:26.040670 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:26.040685 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.042543 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.042552 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:26.044336 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:26.044350 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.046480 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.046487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:26.046692 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:26.046717 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 06:49:26.046818 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 06:49:26.046959 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9330,"mutex_wait_us":2,"thread_start_us":9174,"threads_started":1}
I20260619 06:49:26.047739 4005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:26.027492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:38019 (local address 127.0.0.1:48094)
0619 06:49:26.028452 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:26.028471 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:26.028581 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:26.039867 (+ 11286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:26.039872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:26.039901 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:26.040459 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:26.040474 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.042034 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.042042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:26.044630 (+ 2588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:26.044644 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.046530 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.046540 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:26.046742 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:26.046779 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 06:49:26.046859 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 06:49:26.047012 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":831,"mutex_wait_us":4,"thread_start_us":770,"threads_started":1}
I20260619 06:49:26.048435 4006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:26.028069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48094 (local address 127.16.79.66:38019)
0619 06:49:26.039183 (+ 11114us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:26.039191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:26.039221 (+ 30us) server_negotiation.cc:407] Connection header received
0619 06:49:26.039321 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:26.039330 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:26.039620 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:26.040387 (+ 767us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:26.040645 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.041898 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.044800 (+ 2902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.046237 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.046386 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:26.046583 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:26.047438 (+ 855us) server_negotiation.cc:299] Negotiation successful
0619 06:49:26.048105 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10926,"thread_start_us":10786,"threads_started":1}
I20260619 06:49:26.051249 3926 tablet_service.cc:1467] Processing CreateTablet for tablet 289611f3165b4033ba218ad191a8780c (DEFAULT_TABLE table=test-table [id=5f56b27c3ddb40a8802a88acc7b0ca81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:26.052757 3926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 289611f3165b4033ba218ad191a8780c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:26.052866 3792 tablet_service.cc:1467] Processing CreateTablet for tablet 289611f3165b4033ba218ad191a8780c (DEFAULT_TABLE table=test-table [id=5f56b27c3ddb40a8802a88acc7b0ca81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:26.054160 3792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 289611f3165b4033ba218ad191a8780c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:26.060333 4009 tablet_bootstrap.cc:492] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8: Bootstrap starting.
I20260619 06:49:26.063470 4009 tablet_bootstrap.cc:654] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:26.064556 4009 log.cc:824] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:26.069813 4012 tablet_bootstrap.cc:492] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094: Bootstrap starting.
I20260619 06:49:26.070611 4011 tablet_bootstrap.cc:492] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab: Bootstrap starting.
I20260619 06:49:26.073832 4011 tablet_bootstrap.cc:654] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:26.074031 4012 tablet_bootstrap.cc:654] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:26.074581 4009 tablet_bootstrap.cc:492] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8: No bootstrap required, opened a new log
I20260619 06:49:26.074879 4009 ts_tablet_manager.cc:1397] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260619 06:49:26.074946 4011 log.cc:824] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:26.075249 4012 log.cc:824] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:26.078296 4012 tablet_bootstrap.cc:492] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094: No bootstrap required, opened a new log
I20260619 06:49:26.078332 4011 tablet_bootstrap.cc:492] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab: No bootstrap required, opened a new log
I20260619 06:49:26.078564 4011 ts_tablet_manager.cc:1397] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260619 06:49:26.078560 4012 ts_tablet_manager.cc:1397] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260619 06:49:26.082998 4009 raft_consensus.cc:348] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b67305ab7149a1ad955d29c79e3ec8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36473 } } peers { permanent_uuid: "b881e5df4a8645129f94d46ef0d57eab" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39071 } } peers { permanent_uuid: "21a48b141ddc45448ac66edd5df96094" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38019 } }
I20260619 06:49:26.083374 4009 raft_consensus.cc:374] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:26.083508 4009 raft_consensus.cc:729] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7b67305ab7149a1ad955d29c79e3ec8, State: Initialized, Role: FOLLOWER
I20260619 06:49:26.084117 4009 consensus_queue.cc:260] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b67305ab7149a1ad955d29c79e3ec8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36473 } } peers { permanent_uuid: "b881e5df4a8645129f94d46ef0d57eab" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39071 } } peers { permanent_uuid: "21a48b141ddc45448ac66edd5df96094" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38019 } }
I20260619 06:49:26.086136 4009 ts_tablet_manager.cc:1428] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8: Time spent starting tablet: real 0.011s user 0.009s sys 0.005s
I20260619 06:49:26.089389 4012 raft_consensus.cc:348] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b67305ab7149a1ad955d29c79e3ec8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36473 } } peers { permanent_uuid: "b881e5df4a8645129f94d46ef0d57eab" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39071 } } peers { permanent_uuid: "21a48b141ddc45448ac66edd5df96094" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38019 } }
I20260619 06:49:26.089412 4011 raft_consensus.cc:348] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b67305ab7149a1ad955d29c79e3ec8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36473 } } peers { permanent_uuid: "b881e5df4a8645129f94d46ef0d57eab" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39071 } } peers { permanent_uuid: "21a48b141ddc45448ac66edd5df96094" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38019 } }
I20260619 06:49:26.089857 4012 raft_consensus.cc:374] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:26.089854 4011 raft_consensus.cc:374] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:26.089955 4012 raft_consensus.cc:729] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a48b141ddc45448ac66edd5df96094, State: Initialized, Role: FOLLOWER
I20260619 06:49:26.089955 4011 raft_consensus.cc:729] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b881e5df4a8645129f94d46ef0d57eab, State: Initialized, Role: FOLLOWER
I20260619 06:49:26.090569 4012 consensus_queue.cc:260] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b67305ab7149a1ad955d29c79e3ec8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36473 } } peers { permanent_uuid: "b881e5df4a8645129f94d46ef0d57eab" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39071 } } peers { permanent_uuid: "21a48b141ddc45448ac66edd5df96094" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38019 } }
I20260619 06:49:26.090569 4011 consensus_queue.cc:260] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b67305ab7149a1ad955d29c79e3ec8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36473 } } peers { permanent_uuid: "b881e5df4a8645129f94d46ef0d57eab" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39071 } } peers { permanent_uuid: "21a48b141ddc45448ac66edd5df96094" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38019 } }
I20260619 06:49:26.093130 3991 heartbeater.cc:503] Master 127.16.79.126:42935 was elected leader, sending a full tablet report...
I20260619 06:49:26.093329 3857 heartbeater.cc:503] Master 127.16.79.126:42935 was elected leader, sending a full tablet report...
I20260619 06:49:26.093509 4012 ts_tablet_manager.cc:1428] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094: Time spent starting tablet: real 0.015s user 0.004s sys 0.011s
I20260619 06:49:26.093735 4011 ts_tablet_manager.cc:1428] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
I20260619 06:49:26.114982 4017 raft_consensus.cc:482] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:49:26.115293 4017 raft_consensus.cc:504] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b67305ab7149a1ad955d29c79e3ec8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36473 } } peers { permanent_uuid: "b881e5df4a8645129f94d46ef0d57eab" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39071 } } peers { permanent_uuid: "21a48b141ddc45448ac66edd5df96094" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38019 } }
I20260619 06:49:26.117038 4017 leader_election.cc:288] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7b67305ab7149a1ad955d29c79e3ec8 (127.16.79.65:36473), b881e5df4a8645129f94d46ef0d57eab (127.16.79.67:39071)
I20260619 06:49:26.125599 4008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:26.118644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:56035 (local address 127.16.79.67:39071)
0619 06:49:26.118847 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:26.118853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:26.119037 (+ 184us) server_negotiation.cc:407] Connection header received
0619 06:49:26.119299 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:26.119303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:26.119363 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:26.119470 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:26.119932 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.120908 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.123229 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.124470 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.124608 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:26.124683 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:26.125201 (+ 518us) server_negotiation.cc:299] Negotiation successful
0619 06:49:26.125346 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 06:49:26.125620 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:26.117481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:53847 (local address 127.16.79.65:36473)
0619 06:49:26.117683 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:26.117688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:26.118359 (+ 671us) server_negotiation.cc:407] Connection header received
0619 06:49:26.118554 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:26.118581 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:26.118665 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:26.118771 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:26.119465 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.120519 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.123058 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.124341 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.124525 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:26.124619 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:26.125201 (+ 582us) server_negotiation.cc:299] Negotiation successful
0619 06:49:26.125346 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 06:49:26.125769 4018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:26.117447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:36473 (local address 127.16.79.66:53847)
0619 06:49:26.118225 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:26.118251 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:26.118390 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:26.118888 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:26.118893 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:26.118924 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:26.119333 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:26.119348 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.120661 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.120665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:26.122898 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:26.122914 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.124620 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.124627 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:26.124887 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:26.124979 (+ 92us) client_negotiation.cc:769] Sending connection context
0619 06:49:26.125098 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 06:49:26.125262 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"mutex_wait_us":74,"thread_start_us":425,"threads_started":1}
I20260619 06:49:26.125983 4019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:26.118353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:39071 (local address 127.16.79.66:56035)
0619 06:49:26.118931 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:26.118949 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:26.119161 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:26.119509 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:26.119512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:26.119530 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:26.119785 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:26.119792 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.121023 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.121028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:26.123081 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:26.123091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.124671 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.124676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:26.124928 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:26.124983 (+ 55us) client_negotiation.cc:769] Sending connection context
0619 06:49:26.125098 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 06:49:26.125262 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":66,"spinlock_wait_cycles":27392,"thread_start_us":311,"threads_started":1}
I20260619 06:49:26.126911 3678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289611f3165b4033ba218ad191a8780c" candidate_uuid: "21a48b141ddc45448ac66edd5df96094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7b67305ab7149a1ad955d29c79e3ec8" is_pre_election: true
I20260619 06:49:26.127151 3946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289611f3165b4033ba218ad191a8780c" candidate_uuid: "21a48b141ddc45448ac66edd5df96094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b881e5df4a8645129f94d46ef0d57eab" is_pre_election: true
I20260619 06:49:26.127605 3678 raft_consensus.cc:2454] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a48b141ddc45448ac66edd5df96094 in term 0.
I20260619 06:49:26.127744 3946 raft_consensus.cc:2454] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a48b141ddc45448ac66edd5df96094 in term 0.
I20260619 06:49:26.128477 3747 leader_election.cc:302] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a48b141ddc45448ac66edd5df96094, a7b67305ab7149a1ad955d29c79e3ec8; no voters:
I20260619 06:49:26.128887 4017 raft_consensus.cc:2781] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:49:26.129045 4017 raft_consensus.cc:482] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:49:26.129156 4017 raft_consensus.cc:3037] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:26.131429 4017 raft_consensus.cc:504] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b67305ab7149a1ad955d29c79e3ec8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36473 } } peers { permanent_uuid: "b881e5df4a8645129f94d46ef0d57eab" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39071 } } peers { permanent_uuid: "21a48b141ddc45448ac66edd5df96094" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38019 } }
I20260619 06:49:26.132097 4017 leader_election.cc:288] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [CANDIDATE]: Term 1 election: Requested vote from peers a7b67305ab7149a1ad955d29c79e3ec8 (127.16.79.65:36473), b881e5df4a8645129f94d46ef0d57eab (127.16.79.67:39071)
I20260619 06:49:26.132702 3678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289611f3165b4033ba218ad191a8780c" candidate_uuid: "21a48b141ddc45448ac66edd5df96094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7b67305ab7149a1ad955d29c79e3ec8"
I20260619 06:49:26.132817 3946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289611f3165b4033ba218ad191a8780c" candidate_uuid: "21a48b141ddc45448ac66edd5df96094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b881e5df4a8645129f94d46ef0d57eab"
I20260619 06:49:26.132951 3678 raft_consensus.cc:3037] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:26.133131 3946 raft_consensus.cc:3037] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:26.135265 3678 raft_consensus.cc:2454] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a48b141ddc45448ac66edd5df96094 in term 1.
I20260619 06:49:26.135564 3946 raft_consensus.cc:2454] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a48b141ddc45448ac66edd5df96094 in term 1.
I20260619 06:49:26.135793 3747 leader_election.cc:302] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a48b141ddc45448ac66edd5df96094, a7b67305ab7149a1ad955d29c79e3ec8; no voters:
I20260619 06:49:26.136194 4017 raft_consensus.cc:2781] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:26.136679 4017 raft_consensus.cc:686] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [term 1 LEADER]: Becoming Leader. State: Replica: 21a48b141ddc45448ac66edd5df96094, State: Running, Role: LEADER
I20260619 06:49:26.137192 4017 consensus_queue.cc:237] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b67305ab7149a1ad955d29c79e3ec8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36473 } } peers { permanent_uuid: "b881e5df4a8645129f94d46ef0d57eab" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39071 } } peers { permanent_uuid: "21a48b141ddc45448ac66edd5df96094" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38019 } }
W20260619 06:49:26.140771 3724 tablet.cc:2369] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:49:26.142155 3534 catalog_manager.cc:5495] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 21a48b141ddc45448ac66edd5df96094 (127.16.79.66). New cstate: current_term: 1 leader_uuid: "21a48b141ddc45448ac66edd5df96094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7b67305ab7149a1ad955d29c79e3ec8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b881e5df4a8645129f94d46ef0d57eab" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a48b141ddc45448ac66edd5df96094" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38019 } health_report { overall_health: HEALTHY } } }
I20260619 06:49:26.165087 4006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:26.159180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48110 (local address 127.16.79.66:38019)
0619 06:49:26.159404 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:26.159408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:26.159426 (+ 18us) server_negotiation.cc:407] Connection header received
0619 06:49:26.159515 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:26.159519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:26.159601 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:26.159734 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:49:26.160218 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.160945 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.162588 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.163008 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.163123 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:26.163621 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:49:26.164355 (+ 734us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:49:26.164471 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:26.164668 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 06:49:26.164744 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260619 06:49:26.173826 3946 raft_consensus.cc:1261] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab [term 1 FOLLOWER]: Refusing update from remote peer 21a48b141ddc45448ac66edd5df96094: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:49:26.173851 3678 raft_consensus.cc:1261] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8 [term 1 FOLLOWER]: Refusing update from remote peer 21a48b141ddc45448ac66edd5df96094: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:49:26.174839 4020 consensus_queue.cc:1035] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7b67305ab7149a1ad955d29c79e3ec8" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 36473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:49:26.175230 4017 consensus_queue.cc:1035] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b881e5df4a8645129f94d46ef0d57eab" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 06:49:26.181903 3858 tablet.cc:2369] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:49:26.190685 4023 mvcc.cc:205] Tried to move back new op lower bound from 7298464834237239296 to 7298464834102063104. Current Snapshot: MvccSnapshot[applied={T|T < 7298464834237239296}]
I20260619 06:49:26.205191 3534 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:49:26.205494 3534 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:49:26.209404 3534 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d6858f56424b48d1befcd537d75c1a43: Sending DeleteTablet for 3 replicas of tablet 289611f3165b4033ba218ad191a8780c
I20260619 06:49:26.210631 3658 tablet_service.cc:1514] Processing DeleteTablet for tablet 289611f3165b4033ba218ad191a8780c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:26 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:37140
I20260619 06:49:26.210704 3926 tablet_service.cc:1514] Processing DeleteTablet for tablet 289611f3165b4033ba218ad191a8780c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:26 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:47368
I20260619 06:49:26.211166 3792 tablet_service.cc:1514] Processing DeleteTablet for tablet 289611f3165b4033ba218ad191a8780c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:26 UTC) from {username='kudu', principal='kudu/127.16.79.126@KRBTEST.COM'} at 127.0.0.1:48094
I20260619 06:49:26.212972 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 3595
I20260619 06:49:26.221783 4036 tablet_replica.cc:330] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094: stopping tablet replica
I20260619 06:49:26.221975 4034 tablet_replica.cc:330] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8: stopping tablet replica
I20260619 06:49:26.222555 4034 raft_consensus.cc:2229] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:49:26.223150 4034 raft_consensus.cc:2258] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:26.223142 4036 raft_consensus.cc:2229] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:49:26.224588 4036 raft_consensus.cc:2258] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:26.225507 4034 ts_tablet_manager.cc:1905] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:26.228924 4036 ts_tablet_manager.cc:1905] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:26.232692 4035 tablet_replica.cc:330] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab: stopping tablet replica
I20260619 06:49:26.233142 4035 raft_consensus.cc:2229] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:49:26.233691 4035 raft_consensus.cc:2258] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:26.233809 4034 ts_tablet_manager.cc:1918] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:26.233989 4034 log.cc:1196] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-0/wal/wals/289611f3165b4033ba218ad191a8780c
I20260619 06:49:26.234510 4034 ts_tablet_manager.cc:1939] T 289611f3165b4033ba218ad191a8780c P a7b67305ab7149a1ad955d29c79e3ec8: Deleting consensus metadata
I20260619 06:49:26.236568 3520 catalog_manager.cc:4841] TS a7b67305ab7149a1ad955d29c79e3ec8 (127.16.79.65:36473): tablet 289611f3165b4033ba218ad191a8780c (table test-table [id=5f56b27c3ddb40a8802a88acc7b0ca81]) successfully deleted
Jun 19 06:49:26 dist-test-slave-5w2x krb5kdc[3489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.65@KRBTEST.COM
I20260619 06:49:26.238536 4035 ts_tablet_manager.cc:1905] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:26.239500 4036 ts_tablet_manager.cc:1918] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:26.239756 4036 log.cc:1196] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/289611f3165b4033ba218ad191a8780c
I20260619 06:49:26.240362 4036 ts_tablet_manager.cc:1939] T 289611f3165b4033ba218ad191a8780c P 21a48b141ddc45448ac66edd5df96094: Deleting consensus metadata
I20260619 06:49:26.242241 3521 catalog_manager.cc:4841] TS 21a48b141ddc45448ac66edd5df96094 (127.16.79.66:38019): tablet 289611f3165b4033ba218ad191a8780c (table test-table [id=5f56b27c3ddb40a8802a88acc7b0ca81]) successfully deleted
I20260619 06:49:26.246469 4035 ts_tablet_manager.cc:1918] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:26.246656 4035 log.cc:1196] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/289611f3165b4033ba218ad191a8780c
I20260619 06:49:26.247267 4035 ts_tablet_manager.cc:1939] T 289611f3165b4033ba218ad191a8780c P b881e5df4a8645129f94d46ef0d57eab: Deleting consensus metadata
I20260619 06:49:26.249182 3520 catalog_manager.cc:4841] TS b881e5df4a8645129f94d46ef0d57eab (127.16.79.67:39071): tablet 289611f3165b4033ba218ad191a8780c (table test-table [id=5f56b27c3ddb40a8802a88acc7b0ca81]) successfully deleted
I20260619 06:49:26.249732 4004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:26.214392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37156 (local address 127.16.79.65:36473)
0619 06:49:26.214572 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:26.214579 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:26.224459 (+ 9880us) server_negotiation.cc:407] Connection header received
0619 06:49:26.224767 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:26.224773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:26.224866 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:26.224939 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:26.226978 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.228329 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.230253 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.230937 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.231043 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:26.238572 (+ 7529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:26.238587 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:26.238604 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:26.238654 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:26.245797 (+ 7143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:26.246956 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:26.246970 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:26.246974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:26.247153 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:26.247732 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:26.247740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:26.247746 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:26.248216 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:26.248490 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:26.249117 (+ 627us) server_negotiation.cc:299] Negotiation successful
0619 06:49:26.249414 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:49:26.251096 3718 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:26.344458 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 3595
I20260619 06:49:26.369104 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 3729
Jun 19 06:49:26 dist-test-slave-5w2x krb5kdc[3489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.66@KRBTEST.COM
I20260619 06:49:26.387265 4006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:26.369847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48124 (local address 127.16.79.66:38019)
0619 06:49:26.370034 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:26.370038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:26.370125 (+ 87us) server_negotiation.cc:407] Connection header received
0619 06:49:26.370303 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:26.370307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:26.370369 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:26.370455 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:26.372234 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.373063 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.374930 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.375359 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.375456 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:26.380919 (+ 5463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:26.380935 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:26.380951 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:26.380989 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:26.384127 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:26.384964 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:26.384973 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:26.384976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:26.385202 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:26.385631 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:26.385636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:26.385640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:26.386011 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:26.386370 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:26.386640 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 06:49:26.386865 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:49:26.388041 3852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:26.504042 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 3729
I20260619 06:49:26.522727 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 3863
Jun 19 06:49:26 dist-test-slave-5w2x krb5kdc[3489](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.67@KRBTEST.COM
I20260619 06:49:26.540594 4008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:26.523577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47378 (local address 127.16.79.67:39071)
0619 06:49:26.523780 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:26.523788 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:26.523809 (+ 21us) server_negotiation.cc:407] Connection header received
0619 06:49:26.524018 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:26.524031 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:26.524096 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:26.524267 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:26.525735 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.526497 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.527832 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:26.528434 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:26.528574 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:26.533847 (+ 5273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:26.533865 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:26.533882 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:26.533929 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:26.537368 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:26.538252 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:26.538261 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:26.538263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:26.538426 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:26.538863 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:26.538871 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:26.538880 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:26.539280 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:26.539599 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:26.539946 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 06:49:26.540244 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 06:49:26.541421 3986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:26.632932 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 3863
I20260619 06:49:26.649699 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 3502
I20260619 06:49:26.650918 3564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:26.774001 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 3502
2026-06-19T06:49:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4428 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T06:49:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:26Z Disabled control of system clock
I20260619 06:49:26.853003 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:34621
--webserver_interface=127.16.79.126
--webserver_port=0
--builtin_ntp_servers=127.16.79.84:38151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:34621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 06:49:27.191671 4045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:27.191995 4045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:27.192083 4045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:27.199635 4045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:27.199762 4045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:27.199802 4045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:27.199852 4045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:27.199898 4045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:27.209738 4045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:38151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:34621
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:34621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:27.210994 4045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:27.212745 4045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:27.224628 4051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:27.224767 4050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:27.224736 4053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:27.225634 4045 server_base.cc:1034] running on GCE node
I20260619 06:49:27.226399 4045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:27.227864 4045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:27.229063 4045 hybrid_clock.cc:648] HybridClock initialized: now 1781851767229005 us; error 62 us; skew 500 ppm
I20260619 06:49:27.229453 4045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:49:27.231815 4045 webserver.cc:460] Webserver started at http://127.16.79.126:33423/ using document root <none> and password file <none>
I20260619 06:49:27.232607 4045 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:27.232720 4045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:27.233042 4045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:27.235868 4045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "d43cbe59779649c8b7769c5795592184"
format_stamp: "Formatted at 2026-06-19 06:49:27 on dist-test-slave-5w2x"
server_key: "1c8afb706cd860ab011d05626ce98361"
server_key_iv: "e3c3c187c044140c81b22005d5eeb12e"
server_key_version: "encryptionkey@0"
I20260619 06:49:27.236737 4045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "d43cbe59779649c8b7769c5795592184"
format_stamp: "Formatted at 2026-06-19 06:49:27 on dist-test-slave-5w2x"
server_key: "1c8afb706cd860ab011d05626ce98361"
server_key_iv: "e3c3c187c044140c81b22005d5eeb12e"
server_key_version: "encryptionkey@0"
I20260619 06:49:27.242013 4045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:49:27.245651 4059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:27.247123 4045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:49:27.247340 4045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "d43cbe59779649c8b7769c5795592184"
format_stamp: "Formatted at 2026-06-19 06:49:27 on dist-test-slave-5w2x"
server_key: "1c8afb706cd860ab011d05626ce98361"
server_key_iv: "e3c3c187c044140c81b22005d5eeb12e"
server_key_version: "encryptionkey@0"
I20260619 06:49:27.247570 4045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:27.278762 4045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:27.279672 4045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:27.279980 4045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:27.300290 4045 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:34621
I20260619 06:49:27.300271 4110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:34621 every 8 connection(s)
I20260619 06:49:27.301693 4045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:27.305630 4111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:27.310423 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 4045
I20260619 06:49:27.310613 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:27.311198 16701 external_mini_cluster.cc:1428] Setting key 36a0d15a46f24a812b372f4846c3a94b
I20260619 06:49:27.317757 4111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184: Bootstrap starting.
I20260619 06:49:27.321280 4111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:27.322415 4111 log.cc:824] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:27.326148 4111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184: No bootstrap required, opened a new log
I20260619 06:49:27.326596 4114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:27.313433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41348 (local address 127.16.79.126:34621)
0619 06:49:27.314564 (+ 1131us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:27.314576 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:27.314625 (+ 49us) server_negotiation.cc:407] Connection header received
0619 06:49:27.316592 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:27.316621 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:27.317136 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:27.317560 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:27.318178 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:27.319672 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:27.321139 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:27.321976 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:27.322195 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:27.323569 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:27.323581 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:27.323595 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:27.324344 (+ 749us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:27.324446 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:27.324476 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:27.324603 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:27.324825 (+ 222us) server_negotiation.cc:299] Negotiation successful
0619 06:49:27.325255 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":391,"threads_started":1}
I20260619 06:49:27.334301 4111 raft_consensus.cc:348] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43cbe59779649c8b7769c5795592184" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34621 } }
I20260619 06:49:27.334694 4111 raft_consensus.cc:374] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:27.334774 4111 raft_consensus.cc:729] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d43cbe59779649c8b7769c5795592184, State: Initialized, Role: FOLLOWER
I20260619 06:49:27.335377 4111 consensus_queue.cc:260] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43cbe59779649c8b7769c5795592184" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34621 } }
I20260619 06:49:27.335703 4111 raft_consensus.cc:388] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:27.335860 4111 raft_consensus.cc:482] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:27.336054 4111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:27.338363 4111 raft_consensus.cc:504] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43cbe59779649c8b7769c5795592184" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34621 } }
I20260619 06:49:27.338927 4111 leader_election.cc:302] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d43cbe59779649c8b7769c5795592184; no voters:
I20260619 06:49:27.339545 4111 leader_election.cc:288] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:27.339736 4116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:27.340719 4116 raft_consensus.cc:686] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [term 1 LEADER]: Becoming Leader. State: Replica: d43cbe59779649c8b7769c5795592184, State: Running, Role: LEADER
I20260619 06:49:27.341298 4116 consensus_queue.cc:237] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43cbe59779649c8b7769c5795592184" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34621 } }
I20260619 06:49:27.341842 4111 sys_catalog.cc:564] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:27.343962 4118 sys_catalog.cc:455] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d43cbe59779649c8b7769c5795592184. Latest consensus state: current_term: 1 leader_uuid: "d43cbe59779649c8b7769c5795592184" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43cbe59779649c8b7769c5795592184" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34621 } } }
I20260619 06:49:27.344434 4118 sys_catalog.cc:458] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:27.344333 4117 sys_catalog.cc:455] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d43cbe59779649c8b7769c5795592184" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d43cbe59779649c8b7769c5795592184" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 34621 } } }
I20260619 06:49:27.344907 4117 sys_catalog.cc:458] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:27.346992 4121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:27.352782 4121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:27.362017 4121 catalog_manager.cc:1347] Generated new cluster ID: bb35692c39b24007a891b2a14f1c4ac5
I20260619 06:49:27.362219 4121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:27.380384 4121 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:27.382362 4121 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:27.392099 4121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184: Generated new TSK 0
I20260619 06:49:27.392984 4121 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:27.401283 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.65:0
--local_ip_for_outbound_sockets=127.16.79.65
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:34621
--builtin_ntp_servers=127.16.79.84:38151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 06:49:27.737077 4135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:27.737370 4135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:27.737550 4135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:27.745110 4135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:27.745216 4135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:27.745388 4135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.65
I20260619 06:49:27.755234 4135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:38151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.16.79.65
--webserver_port=0
--tserver_master_addrs=127.16.79.126:34621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.65
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:27.756600 4135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:27.758445 4135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:27.771839 4140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:27.771862 4143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:27.771924 4141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:27.777863 4135 server_base.cc:1034] running on GCE node
I20260619 06:49:27.778564 4135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:27.779551 4135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:27.780766 4135 hybrid_clock.cc:648] HybridClock initialized: now 1781851767780714 us; error 41 us; skew 500 ppm
I20260619 06:49:27.781096 4135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:49:27.783412 4135 webserver.cc:460] Webserver started at http://127.16.79.65:35077/ using document root <none> and password file <none>
I20260619 06:49:27.784204 4135 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:27.784327 4135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:27.784675 4135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:27.787613 4135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/data/instance:
uuid: "94194c6173074fd39f0f898368332177"
format_stamp: "Formatted at 2026-06-19 06:49:27 on dist-test-slave-5w2x"
server_key: "569db973eb06b324e29b87526010aef8"
server_key_iv: "c26f5ae7a769fa94f3b534f55aaaef01"
server_key_version: "encryptionkey@0"
I20260619 06:49:27.788520 4135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance:
uuid: "94194c6173074fd39f0f898368332177"
format_stamp: "Formatted at 2026-06-19 06:49:27 on dist-test-slave-5w2x"
server_key: "569db973eb06b324e29b87526010aef8"
server_key_iv: "c26f5ae7a769fa94f3b534f55aaaef01"
server_key_version: "encryptionkey@0"
I20260619 06:49:27.793980 4135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:49:27.797638 4149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:27.799088 4135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260619 06:49:27.799266 4135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/wal
uuid: "94194c6173074fd39f0f898368332177"
format_stamp: "Formatted at 2026-06-19 06:49:27 on dist-test-slave-5w2x"
server_key: "569db973eb06b324e29b87526010aef8"
server_key_iv: "c26f5ae7a769fa94f3b534f55aaaef01"
server_key_version: "encryptionkey@0"
I20260619 06:49:27.799502 4135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:27.811866 4135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:27.812786 4135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:27.813097 4135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:27.814107 4135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:27.815662 4135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:27.815809 4135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:27.815935 4135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:27.816021 4135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:27.850802 4135 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.65:43225
I20260619 06:49:27.850834 4261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.65:43225 every 8 connection(s)
I20260619 06:49:27.852348 4135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/data/info.pb
I20260619 06:49:27.856397 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 4135
I20260619 06:49:27.856576 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-0/wal/instance
I20260619 06:49:27.857085 16701 external_mini_cluster.cc:1428] Setting key 7cb79359c12c990ec8b1ad784a3a84d2
I20260619 06:49:27.859688 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.66:0
--local_ip_for_outbound_sockets=127.16.79.66
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:34621
--builtin_ntp_servers=127.16.79.84:38151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 06:49:27.865422 4264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:27.855046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.65:32943 (local address 127.16.79.126:34621)
0619 06:49:27.855491 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:27.855495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:27.856891 (+ 1396us) server_negotiation.cc:407] Connection header received
0619 06:49:27.858779 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:27.858784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:27.858874 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:27.858986 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:27.860175 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:27.860886 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:27.862493 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:27.862913 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:27.863056 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:27.864540 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:27.864544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:27.864547 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:27.864566 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:27.864622 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:27.864634 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:27.864730 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:27.864950 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 06:49:27.865064 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":225,"threads_started":1}
I20260619 06:49:27.866427 4265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:27.855679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:34621 (local address 127.16.79.65:32943)
0619 06:49:27.856738 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:27.856783 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:27.858496 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:27.859222 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:27.859230 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:27.859329 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:27.859981 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:27.860000 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:27.861039 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:27.861045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:27.862342 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:27.862355 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:27.863151 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:27.863158 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:27.863383 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:27.864213 (+ 830us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:49:27.864215 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:27.864264 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:49:27.864344 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:27.864730 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:27.864744 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:27.864754 (+ 10us) client_negotiation.cc:769] Sending connection context
0619 06:49:27.864946 (+ 192us) client_negotiation.cc:240] Negotiation successful
0619 06:49:27.865364 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":285,"threads_started":1}
I20260619 06:49:27.867996 4262 heartbeater.cc:347] Connected to a master server at 127.16.79.126:34621
I20260619 06:49:27.868368 4262 heartbeater.cc:464] Registering TS with master...
I20260619 06:49:27.869257 4262 heartbeater.cc:511] Master 127.16.79.126:34621 requested a full tablet report, sending...
I20260619 06:49:27.871438 4076 ts_manager.cc:194] Registered new tserver with Master: 94194c6173074fd39f0f898368332177 (127.16.79.65:43225)
I20260619 06:49:27.873041 4076 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.79.65:32943
W20260619 06:49:28.196345 4266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:28.196632 4266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:28.196779 4266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:28.204272 4266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:28.204380 4266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:28.204555 4266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.66
I20260619 06:49:28.213837 4266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:38151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.16.79.66
--webserver_port=0
--tserver_master_addrs=127.16.79.126:34621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.66
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:28.215515 4266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:28.217389 4266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:28.230755 4271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:28.230785 4272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:28.231199 4274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:28.232043 4266 server_base.cc:1034] running on GCE node
I20260619 06:49:28.232728 4266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:28.233659 4266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:28.234876 4266 hybrid_clock.cc:648] HybridClock initialized: now 1781851768234809 us; error 57 us; skew 500 ppm
I20260619 06:49:28.235239 4266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:49:28.237540 4266 webserver.cc:460] Webserver started at http://127.16.79.66:33935/ using document root <none> and password file <none>
I20260619 06:49:28.238215 4266 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:28.238300 4266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:28.238554 4266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:28.241624 4266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/data/instance:
uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f"
format_stamp: "Formatted at 2026-06-19 06:49:28 on dist-test-slave-5w2x"
server_key: "c885a6066ac289175086c8ab4b3082be"
server_key_iv: "9a11d9b84e6c0e7c0592a68e83eb36d4"
server_key_version: "encryptionkey@0"
I20260619 06:49:28.242532 4266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance:
uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f"
format_stamp: "Formatted at 2026-06-19 06:49:28 on dist-test-slave-5w2x"
server_key: "c885a6066ac289175086c8ab4b3082be"
server_key_iv: "9a11d9b84e6c0e7c0592a68e83eb36d4"
server_key_version: "encryptionkey@0"
I20260619 06:49:28.248463 4266 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260619 06:49:28.252465 4280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:28.254007 4266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:49:28.254184 4266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/wal
uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f"
format_stamp: "Formatted at 2026-06-19 06:49:28 on dist-test-slave-5w2x"
server_key: "c885a6066ac289175086c8ab4b3082be"
server_key_iv: "9a11d9b84e6c0e7c0592a68e83eb36d4"
server_key_version: "encryptionkey@0"
I20260619 06:49:28.254400 4266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:28.282212 4266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:28.283110 4266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:28.283408 4266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:28.284493 4266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:28.286185 4266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:28.286326 4266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:28.286470 4266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:28.286581 4266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:28.325922 4266 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.66:38705
I20260619 06:49:28.325982 4392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.66:38705 every 8 connection(s)
I20260619 06:49:28.327456 4266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/data/info.pb
I20260619 06:49:28.337774 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 4266
I20260619 06:49:28.337965 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/wal/instance
I20260619 06:49:28.338555 16701 external_mini_cluster.cc:1428] Setting key e2af8c2c40e8a33d7aace281611aa894
I20260619 06:49:28.340637 4264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:28.330085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:45095 (local address 127.16.79.126:34621)
0619 06:49:28.330357 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:28.330364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:28.331774 (+ 1410us) server_negotiation.cc:407] Connection header received
0619 06:49:28.333584 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:28.333589 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:28.333660 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:28.333759 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:28.335037 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.335795 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.337457 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.338124 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.338274 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:28.339775 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:28.339780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:28.339783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:28.339802 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:28.339836 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:28.339849 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:28.339953 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:28.340213 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 06:49:28.340315 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260619 06:49:28.341580 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.16.79.67:0
--local_ip_for_outbound_sockets=127.16.79.67
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:34621
--builtin_ntp_servers=127.16.79.84:38151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 06:49:28.341922 4395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:28.330724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:34621 (local address 127.16.79.66:45095)
0619 06:49:28.331600 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:28.331648 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:28.333301 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:28.333997 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:28.334006 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:28.334107 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:28.334775 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:28.334795 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.335940 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.335943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:28.337284 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:28.337297 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.338367 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.338372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:28.338597 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:28.339375 (+ 778us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:49:28.339380 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:28.339449 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:49:28.339558 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:28.339951 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:28.339963 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:28.339974 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 06:49:28.340201 (+ 227us) client_negotiation.cc:240] Negotiation successful
0619 06:49:28.340624 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":261,"threads_started":1}
I20260619 06:49:28.343609 4393 heartbeater.cc:347] Connected to a master server at 127.16.79.126:34621
I20260619 06:49:28.343931 4393 heartbeater.cc:464] Registering TS with master...
I20260619 06:49:28.344766 4393 heartbeater.cc:511] Master 127.16.79.126:34621 requested a full tablet report, sending...
I20260619 06:49:28.346287 4076 ts_manager.cc:194] Registered new tserver with Master: cb6fcac69ea44ee0a9c6b9766f4b8b6f (127.16.79.66:38705)
I20260619 06:49:28.347558 4076 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.79.66:45095
W20260619 06:49:28.678397 4396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:28.678700 4396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:28.678853 4396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:28.686450 4396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:28.686560 4396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:28.686730 4396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.16.79.67
I20260619 06:49:28.696194 4396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:38151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.16.79.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.16.79.67
--webserver_port=0
--tserver_master_addrs=127.16.79.126:34621
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.16.79.67
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:28.697561 4396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:28.699326 4396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:28.712685 4402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:28.712685 4404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:28.712719 4401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:28.713769 4396 server_base.cc:1034] running on GCE node
I20260619 06:49:28.714425 4396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:28.715410 4396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:28.716620 4396 hybrid_clock.cc:648] HybridClock initialized: now 1781851768716549 us; error 67 us; skew 500 ppm
I20260619 06:49:28.716972 4396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260619 06:49:28.719444 4396 webserver.cc:460] Webserver started at http://127.16.79.67:41587/ using document root <none> and password file <none>
I20260619 06:49:28.720238 4396 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:28.720355 4396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:28.720691 4396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:28.723632 4396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/data/instance:
uuid: "15422b3e22e24ffb8424459f175ba32a"
format_stamp: "Formatted at 2026-06-19 06:49:28 on dist-test-slave-5w2x"
server_key: "f83ef2eef2c39ddc71d3100586a1e431"
server_key_iv: "a40939fd5a67e757c0c10cf880e4e9e2"
server_key_version: "encryptionkey@0"
I20260619 06:49:28.724514 4396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance:
uuid: "15422b3e22e24ffb8424459f175ba32a"
format_stamp: "Formatted at 2026-06-19 06:49:28 on dist-test-slave-5w2x"
server_key: "f83ef2eef2c39ddc71d3100586a1e431"
server_key_iv: "a40939fd5a67e757c0c10cf880e4e9e2"
server_key_version: "encryptionkey@0"
I20260619 06:49:28.729715 4396 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260619 06:49:28.733222 4410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:28.734592 4396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:49:28.734781 4396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/wal
uuid: "15422b3e22e24ffb8424459f175ba32a"
format_stamp: "Formatted at 2026-06-19 06:49:28 on dist-test-slave-5w2x"
server_key: "f83ef2eef2c39ddc71d3100586a1e431"
server_key_iv: "a40939fd5a67e757c0c10cf880e4e9e2"
server_key_version: "encryptionkey@0"
I20260619 06:49:28.735010 4396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:28.752352 4396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:28.753196 4396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:28.753501 4396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:28.754472 4396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 06:49:28.755913 4396 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 06:49:28.756044 4396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:28.756191 4396 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 06:49:28.756289 4396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260619 06:49:28.790251 4396 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.67:39449
I20260619 06:49:28.790328 4522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.67:39449 every 8 connection(s)
I20260619 06:49:28.791576 4396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/data/info.pb
I20260619 06:49:28.798475 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 4396
I20260619 06:49:28.798688 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/wal/instance
I20260619 06:49:28.799137 16701 external_mini_cluster.cc:1428] Setting key d214d8c4d8e9b7f65bf93a2fac8bce1b
I20260619 06:49:28.804010 4264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:28.794160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.67:49847 (local address 127.16.79.126:34621)
0619 06:49:28.794379 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:28.794385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:28.795733 (+ 1348us) server_negotiation.cc:407] Connection header received
0619 06:49:28.797547 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:28.797552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:28.797623 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:28.797712 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:28.798825 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.799633 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.801230 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.801756 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.801887 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:28.803310 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:28.803315 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:28.803318 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:28.803336 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:28.803369 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:28.803380 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:28.803458 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:28.803654 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 06:49:28.803731 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 06:49:28.805074 4525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:28.794702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:34621 (local address 127.16.79.67:49847)
0619 06:49:28.795576 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:28.795614 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:28.797282 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:28.797926 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:28.797934 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:28.798021 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:49:28.798635 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:28.798661 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.799780 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.799786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:28.801086 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:28.801098 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.801971 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.801980 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:28.802217 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:28.802909 (+ 692us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 06:49:28.802912 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:49:28.802951 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 06:49:28.803025 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:49:28.803471 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:49:28.803479 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:49:28.803484 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 06:49:28.803629 (+ 145us) client_negotiation.cc:240] Negotiation successful
0619 06:49:28.803973 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":263,"threads_started":1}
I20260619 06:49:28.806468 4523 heartbeater.cc:347] Connected to a master server at 127.16.79.126:34621
I20260619 06:49:28.806735 4523 heartbeater.cc:464] Registering TS with master...
I20260619 06:49:28.807454 4523 heartbeater.cc:511] Master 127.16.79.126:34621 requested a full tablet report, sending...
I20260619 06:49:28.808820 4076 ts_manager.cc:194] Registered new tserver with Master: 15422b3e22e24ffb8424459f175ba32a (127.16.79.67:39449)
I20260619 06:49:28.809585 4076 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.16.79.67:49847
I20260619 06:49:28.813987 16701 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 06:49:28.824410 4264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:28.818203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41362 (local address 127.16.79.126:34621)
0619 06:49:28.818398 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:28.818402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:28.818854 (+ 452us) server_negotiation.cc:407] Connection header received
0619 06:49:28.818991 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:28.818995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:28.819070 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:28.819167 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:28.819645 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.820514 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.821983 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.822548 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.822649 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:28.823687 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:28.823692 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:28.823695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:28.823713 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:28.823745 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:28.823757 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:28.823829 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:28.823985 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 06:49:28.824062 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:49:28.828504 4076 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 06:49:28.835414 4076 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 06:49:28.876753 4262 heartbeater.cc:503] Master 127.16.79.126:34621 was elected leader, sending a full tablet report...
I20260619 06:49:28.877907 4537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:28.849800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60028 (local address 127.16.79.66:38705)
0619 06:49:28.862508 (+ 12708us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:28.862521 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:28.866371 (+ 3850us) server_negotiation.cc:407] Connection header received
0619 06:49:28.866733 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:28.866746 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:28.867069 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:28.867259 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:28.868431 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.870484 (+ 2053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.874282 (+ 3798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.876073 (+ 1791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.876256 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:28.876521 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:28.877316 (+ 795us) server_negotiation.cc:299] Negotiation successful
0619 06:49:28.877488 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12455,"thread_start_us":12735,"threads_started":1}
I20260619 06:49:28.878786 4535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:28.849530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.66:38705 (local address 127.0.0.1:60028)
0619 06:49:28.866244 (+ 16714us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:28.866274 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:28.866489 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:28.867643 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:28.867650 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:28.867685 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:28.868184 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:28.868200 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.870753 (+ 2553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.870761 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:28.873974 (+ 3213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:28.874007 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.876659 (+ 2652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.876665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:28.876950 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:28.877083 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 06:49:28.878206 (+ 1123us) client_negotiation.cc:240] Negotiation successful
0619 06:49:28.878431 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16563,"mutex_wait_us":89,"thread_start_us":13384,"threads_started":1}
I20260619 06:49:28.879140 4536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:28.849300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:43225 (local address 127.0.0.1:33440)
0619 06:49:28.863419 (+ 14119us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:28.863481 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:28.863732 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:28.865502 (+ 1770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:28.865509 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:28.865538 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:28.865890 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:28.865902 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.868482 (+ 2580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.868491 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:28.874349 (+ 5858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:28.874364 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.878318 (+ 3954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.878324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:28.878509 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:28.878545 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 06:49:28.878639 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 06:49:28.878787 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13783,"mutex_wait_us":2558,"thread_start_us":16600,"threads_started":1}
I20260619 06:49:28.879298 4538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:28.849816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33440 (local address 127.16.79.65:43225)
0619 06:49:28.864638 (+ 14822us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:28.864650 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:28.864690 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:49:28.864818 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:28.864830 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:28.865225 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:28.865485 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:28.866083 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.868284 (+ 2201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.874977 (+ 6693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.877621 (+ 2644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.877751 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:28.878651 (+ 900us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:28.878732 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 06:49:28.878907 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14620,"thread_start_us":14508,"threads_started":1}
I20260619 06:49:28.883136 4328 tablet_service.cc:1467] Processing CreateTablet for tablet 856d6501fe6d47b19e55f42e167c5cc6 (DEFAULT_TABLE table=test-table [id=504ed2781bc645f1ba97984a19ffb7e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:28.883203 4197 tablet_service.cc:1467] Processing CreateTablet for tablet 856d6501fe6d47b19e55f42e167c5cc6 (DEFAULT_TABLE table=test-table [id=504ed2781bc645f1ba97984a19ffb7e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:28.884215 4539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:28.866379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:39449 (local address 127.0.0.1:46662)
0619 06:49:28.874975 (+ 8596us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:28.874996 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:28.875113 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:28.876707 (+ 1594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:28.876712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:28.876741 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:28.877373 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:28.877386 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.879386 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.879391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:28.881491 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:28.881509 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.883360 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.883366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:28.883531 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:28.883566 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 06:49:28.883674 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 06:49:28.883811 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8424,"mutex_wait_us":151,"thread_start_us":8280,"threads_started":1}
I20260619 06:49:28.884318 4540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:28.866758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46662 (local address 127.16.79.67:39449)
0619 06:49:28.875709 (+ 8951us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:28.875719 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:28.875764 (+ 45us) server_negotiation.cc:407] Connection header received
0619 06:49:28.875883 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:28.875892 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:28.876214 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:28.876420 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:28.877566 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.879214 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.881739 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:28.883138 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:28.883292 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:28.883591 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:28.883733 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 06:49:28.883896 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8761,"thread_start_us":8595,"threads_started":1}
I20260619 06:49:28.884848 4197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 856d6501fe6d47b19e55f42e167c5cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:28.885004 4328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 856d6501fe6d47b19e55f42e167c5cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:28.888460 4458 tablet_service.cc:1467] Processing CreateTablet for tablet 856d6501fe6d47b19e55f42e167c5cc6 (DEFAULT_TABLE table=test-table [id=504ed2781bc645f1ba97984a19ffb7e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 06:49:28.890105 4458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 856d6501fe6d47b19e55f42e167c5cc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:28.897195 4541 tablet_bootstrap.cc:492] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f: Bootstrap starting.
I20260619 06:49:28.899539 4542 tablet_bootstrap.cc:492] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177: Bootstrap starting.
I20260619 06:49:28.901088 4541 tablet_bootstrap.cc:654] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:28.902443 4541 log.cc:824] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:28.903355 4542 tablet_bootstrap.cc:654] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:28.904779 4542 log.cc:824] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:28.905514 4544 tablet_bootstrap.cc:492] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a: Bootstrap starting.
I20260619 06:49:28.906376 4541 tablet_bootstrap.cc:492] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f: No bootstrap required, opened a new log
I20260619 06:49:28.906731 4541 ts_tablet_manager.cc:1397] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260619 06:49:28.909785 4544 tablet_bootstrap.cc:654] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:28.910986 4542 tablet_bootstrap.cc:492] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177: No bootstrap required, opened a new log
I20260619 06:49:28.911026 4544 log.cc:824] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:28.911289 4542 ts_tablet_manager.cc:1397] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260619 06:49:28.914531 4541 raft_consensus.cc:348] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94194c6173074fd39f0f898368332177" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 43225 } } peers { permanent_uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38705 } } peers { permanent_uuid: "15422b3e22e24ffb8424459f175ba32a" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39449 } }
I20260619 06:49:28.915064 4541 raft_consensus.cc:374] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:28.915216 4541 raft_consensus.cc:729] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb6fcac69ea44ee0a9c6b9766f4b8b6f, State: Initialized, Role: FOLLOWER
I20260619 06:49:28.916121 4541 consensus_queue.cc:260] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94194c6173074fd39f0f898368332177" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 43225 } } peers { permanent_uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38705 } } peers { permanent_uuid: "15422b3e22e24ffb8424459f175ba32a" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39449 } }
I20260619 06:49:28.917523 4393 heartbeater.cc:503] Master 127.16.79.126:34621 was elected leader, sending a full tablet report...
I20260619 06:49:28.919478 4541 ts_tablet_manager.cc:1428] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f: Time spent starting tablet: real 0.013s user 0.001s sys 0.010s
I20260619 06:49:28.923872 4542 raft_consensus.cc:348] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94194c6173074fd39f0f898368332177" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 43225 } } peers { permanent_uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38705 } } peers { permanent_uuid: "15422b3e22e24ffb8424459f175ba32a" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39449 } }
I20260619 06:49:28.924436 4542 raft_consensus.cc:374] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:28.924544 4542 raft_consensus.cc:729] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94194c6173074fd39f0f898368332177, State: Initialized, Role: FOLLOWER
I20260619 06:49:28.924993 4544 tablet_bootstrap.cc:492] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a: No bootstrap required, opened a new log
I20260619 06:49:28.925318 4544 ts_tablet_manager.cc:1397] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.005s
I20260619 06:49:28.925216 4542 consensus_queue.cc:260] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94194c6173074fd39f0f898368332177" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 43225 } } peers { permanent_uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38705 } } peers { permanent_uuid: "15422b3e22e24ffb8424459f175ba32a" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39449 } }
I20260619 06:49:28.926817 4542 ts_tablet_manager.cc:1428] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177: Time spent starting tablet: real 0.015s user 0.016s sys 0.001s
I20260619 06:49:28.934134 4544 raft_consensus.cc:348] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94194c6173074fd39f0f898368332177" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 43225 } } peers { permanent_uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38705 } } peers { permanent_uuid: "15422b3e22e24ffb8424459f175ba32a" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39449 } }
I20260619 06:49:28.934523 4544 raft_consensus.cc:374] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:28.934602 4544 raft_consensus.cc:729] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15422b3e22e24ffb8424459f175ba32a, State: Initialized, Role: FOLLOWER
I20260619 06:49:28.935202 4544 consensus_queue.cc:260] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94194c6173074fd39f0f898368332177" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 43225 } } peers { permanent_uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38705 } } peers { permanent_uuid: "15422b3e22e24ffb8424459f175ba32a" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39449 } }
I20260619 06:49:28.936249 4523 heartbeater.cc:503] Master 127.16.79.126:34621 was elected leader, sending a full tablet report...
I20260619 06:49:28.936928 4544 ts_tablet_manager.cc:1428] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
W20260619 06:49:29.043377 4524 tablet.cc:2369] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:49:29.079726 4394 tablet.cc:2369] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 06:49:29.104835 4263 tablet.cc:2369] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:49:29.226151 4547 raft_consensus.cc:482] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 06:49:29.226580 4547 raft_consensus.cc:504] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94194c6173074fd39f0f898368332177" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 43225 } } peers { permanent_uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38705 } } peers { permanent_uuid: "15422b3e22e24ffb8424459f175ba32a" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39449 } }
I20260619 06:49:29.228461 4547 leader_election.cc:288] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94194c6173074fd39f0f898368332177 (127.16.79.65:43225), 15422b3e22e24ffb8424459f175ba32a (127.16.79.67:39449)
I20260619 06:49:29.235863 4538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:29.228875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:43367 (local address 127.16.79.65:43225)
0619 06:49:29.229068 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:29.229073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:29.229664 (+ 591us) server_negotiation.cc:407] Connection header received
0619 06:49:29.230000 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:29.230009 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:29.230095 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:29.230164 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:29.231058 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:29.232209 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:29.234142 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:29.234962 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:29.235148 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:29.235221 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:29.235495 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 06:49:29.235626 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 06:49:29.235955 4550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:29.228849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:43225 (local address 127.16.79.66:43367)
0619 06:49:29.229505 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:29.229527 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:29.229803 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:29.230400 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:29.230405 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:29.230431 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:29.230896 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:29.230910 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:29.232365 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:29.232369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:29.233905 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:29.233921 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:29.235169 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:29.235175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:29.235333 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:29.235356 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 06:49:29.235464 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 06:49:29.235603 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":365,"threads_started":1}
I20260619 06:49:29.236348 4551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:29.229641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.67:39449 (local address 127.16.79.66:46303)
0619 06:49:29.230550 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:29.230565 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 06:49:29.230729 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:49:29.231273 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:49:29.231276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:49:29.231295 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 06:49:29.231541 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:29.231548 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:29.232912 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:29.232918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:29.234453 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:49:29.234465 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:49:29.235565 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:49:29.235571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:49:29.235748 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:29.235770 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 06:49:29.235870 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 06:49:29.236001 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"thread_start_us":596,"threads_started":1}
I20260619 06:49:29.236676 4540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:29.229789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.16.79.66:46303 (local address 127.16.79.67:39449)
0619 06:49:29.230018 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:29.230026 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:29.230670 (+ 644us) server_negotiation.cc:407] Connection header received
0619 06:49:29.230881 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:29.230893 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:29.230955 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:29.231069 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 06:49:29.231700 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:29.232776 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:29.234636 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:29.235373 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:29.235567 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:29.235637 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:29.235908 (+ 271us) server_negotiation.cc:299] Negotiation successful
0619 06:49:29.236036 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 06:49:29.237241 4217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "856d6501fe6d47b19e55f42e167c5cc6" candidate_uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94194c6173074fd39f0f898368332177" is_pre_election: true
I20260619 06:49:29.237560 4478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "856d6501fe6d47b19e55f42e167c5cc6" candidate_uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15422b3e22e24ffb8424459f175ba32a" is_pre_election: true
I20260619 06:49:29.237892 4217 raft_consensus.cc:2454] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb6fcac69ea44ee0a9c6b9766f4b8b6f in term 0.
I20260619 06:49:29.238086 4478 raft_consensus.cc:2454] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb6fcac69ea44ee0a9c6b9766f4b8b6f in term 0.
I20260619 06:49:29.238674 4284 leader_election.cc:302] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94194c6173074fd39f0f898368332177, cb6fcac69ea44ee0a9c6b9766f4b8b6f; no voters:
I20260619 06:49:29.239048 4547 raft_consensus.cc:2781] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 06:49:29.239252 4547 raft_consensus.cc:482] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 06:49:29.239385 4547 raft_consensus.cc:3037] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:29.241772 4547 raft_consensus.cc:504] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94194c6173074fd39f0f898368332177" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 43225 } } peers { permanent_uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38705 } } peers { permanent_uuid: "15422b3e22e24ffb8424459f175ba32a" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39449 } }
I20260619 06:49:29.242492 4547 leader_election.cc:288] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [CANDIDATE]: Term 1 election: Requested vote from peers 94194c6173074fd39f0f898368332177 (127.16.79.65:43225), 15422b3e22e24ffb8424459f175ba32a (127.16.79.67:39449)
I20260619 06:49:29.243086 4217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "856d6501fe6d47b19e55f42e167c5cc6" candidate_uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94194c6173074fd39f0f898368332177"
I20260619 06:49:29.243134 4478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "856d6501fe6d47b19e55f42e167c5cc6" candidate_uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15422b3e22e24ffb8424459f175ba32a"
I20260619 06:49:29.243366 4478 raft_consensus.cc:3037] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:29.243355 4217 raft_consensus.cc:3037] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:29.246793 4217 raft_consensus.cc:2454] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb6fcac69ea44ee0a9c6b9766f4b8b6f in term 1.
I20260619 06:49:29.246793 4478 raft_consensus.cc:2454] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb6fcac69ea44ee0a9c6b9766f4b8b6f in term 1.
I20260619 06:49:29.247305 4284 leader_election.cc:302] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94194c6173074fd39f0f898368332177, cb6fcac69ea44ee0a9c6b9766f4b8b6f; no voters:
I20260619 06:49:29.247587 4547 raft_consensus.cc:2781] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:29.248107 4547 raft_consensus.cc:686] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [term 1 LEADER]: Becoming Leader. State: Replica: cb6fcac69ea44ee0a9c6b9766f4b8b6f, State: Running, Role: LEADER
I20260619 06:49:29.248728 4547 consensus_queue.cc:237] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94194c6173074fd39f0f898368332177" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 43225 } } peers { permanent_uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38705 } } peers { permanent_uuid: "15422b3e22e24ffb8424459f175ba32a" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39449 } }
I20260619 06:49:29.253717 4076 catalog_manager.cc:5495] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f reported cstate change: term changed from 0 to 1, leader changed from <none> to cb6fcac69ea44ee0a9c6b9766f4b8b6f (127.16.79.66). New cstate: current_term: 1 leader_uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94194c6173074fd39f0f898368332177" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 43225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb6fcac69ea44ee0a9c6b9766f4b8b6f" member_type: VOTER last_known_addr { host: "127.16.79.66" port: 38705 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15422b3e22e24ffb8424459f175ba32a" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39449 } health_report { overall_health: UNKNOWN } } }
I20260619 06:49:29.332701 4537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:29.326593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60042 (local address 127.16.79.66:38705)
0619 06:49:29.326854 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:29.326865 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:29.327430 (+ 565us) server_negotiation.cc:407] Connection header received
0619 06:49:29.327675 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:29.327679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:29.327778 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:29.327937 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:49:29.328396 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:29.329134 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:29.330522 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:29.330987 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:29.331128 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:29.331607 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:49:29.332084 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:49:29.332287 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:29.332339 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 06:49:29.332417 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:49:29.342224 4478 raft_consensus.cc:1261] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a [term 1 FOLLOWER]: Refusing update from remote peer cb6fcac69ea44ee0a9c6b9766f4b8b6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:49:29.342411 4217 raft_consensus.cc:1261] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177 [term 1 FOLLOWER]: Refusing update from remote peer cb6fcac69ea44ee0a9c6b9766f4b8b6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:49:29.343326 4552 consensus_queue.cc:1035] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "15422b3e22e24ffb8424459f175ba32a" member_type: VOTER last_known_addr { host: "127.16.79.67" port: 39449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:49:29.343801 4547 consensus_queue.cc:1035] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "94194c6173074fd39f0f898368332177" member_type: VOTER last_known_addr { host: "127.16.79.65" port: 43225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:49:29.368552 4557 mvcc.cc:205] Tried to move back new op lower bound from 7298464847215714304 to 7298464846846689280. Current Snapshot: MvccSnapshot[applied={T|T < 7298464847215714304 or (T in {7298464847215714304})}]
I20260619 06:49:29.376394 4076 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:49:29.376708 4076 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 06:49:29.380542 4076 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d43cbe59779649c8b7769c5795592184: Sending DeleteTablet for 3 replicas of tablet 856d6501fe6d47b19e55f42e167c5cc6
I20260619 06:49:29.381832 4197 tablet_service.cc:1514] Processing DeleteTablet for tablet 856d6501fe6d47b19e55f42e167c5cc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:29 UTC) from {username='slave'} at 127.0.0.1:33440
I20260619 06:49:29.381850 4328 tablet_service.cc:1514] Processing DeleteTablet for tablet 856d6501fe6d47b19e55f42e167c5cc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:29 UTC) from {username='slave'} at 127.0.0.1:60028
I20260619 06:49:29.382230 4458 tablet_service.cc:1514] Processing DeleteTablet for tablet 856d6501fe6d47b19e55f42e167c5cc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 06:49:29 UTC) from {username='slave'} at 127.0.0.1:46662
I20260619 06:49:29.383955 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 4135
I20260619 06:49:29.395090 4567 tablet_replica.cc:330] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f: stopping tablet replica
I20260619 06:49:29.395694 4567 raft_consensus.cc:2229] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [term 1 LEADER]: Raft consensus shutting down.
I20260619 06:49:29.396528 4567 raft_consensus.cc:2258] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:29.397435 4568 tablet_replica.cc:330] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a: stopping tablet replica
I20260619 06:49:29.397948 4568 raft_consensus.cc:2229] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:49:29.398576 4568 raft_consensus.cc:2258] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:29.401626 4567 ts_tablet_manager.cc:1905] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:29.409685 4567 ts_tablet_manager.cc:1918] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:29.409965 4567 log.cc:1196] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-1/wal/wals/856d6501fe6d47b19e55f42e167c5cc6
I20260619 06:49:29.410313 4568 ts_tablet_manager.cc:1905] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:29.410799 4567 ts_tablet_manager.cc:1939] T 856d6501fe6d47b19e55f42e167c5cc6 P cb6fcac69ea44ee0a9c6b9766f4b8b6f: Deleting consensus metadata
I20260619 06:49:29.411345 4566 tablet_replica.cc:330] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177: stopping tablet replica
I20260619 06:49:29.411816 4566 raft_consensus.cc:2229] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 06:49:29.412360 4566 raft_consensus.cc:2258] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 06:49:29.412927 4060 catalog_manager.cc:4841] TS cb6fcac69ea44ee0a9c6b9766f4b8b6f (127.16.79.66:38705): tablet 856d6501fe6d47b19e55f42e167c5cc6 (table test-table [id=504ed2781bc645f1ba97984a19ffb7e0]) successfully deleted
I20260619 06:49:29.414804 4566 ts_tablet_manager.cc:1905] T 856d6501fe6d47b19e55f42e167c5cc6 P 94194c6173074fd39f0f898368332177: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 06:49:29.418304 4538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:29.386169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33452 (local address 127.16.79.65:43225)
0619 06:49:29.386401 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:29.386411 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:29.401468 (+ 15057us) server_negotiation.cc:407] Connection header received
0619 06:49:29.402020 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:29.402026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:29.402167 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:29.402260 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:29.403023 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:29.404257 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:29.407339 (+ 3082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:29.408044 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:29.408167 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:29.416518 (+ 8351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:29.416529 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:29.416547 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:29.417341 (+ 794us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:29.417406 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:29.417434 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:29.417534 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:29.417813 (+ 279us) server_negotiation.cc:299] Negotiation successful
0619 06:49:29.417935 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 06:49:29.418759 4568 ts_tablet_manager.cc:1918] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 06:49:29.418972 4568 log.cc:1196] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a: Deleting WAL directory at /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781851451399607-16701-0/minicluster-data/ts-2/wal/wals/856d6501fe6d47b19e55f42e167c5cc6
I20260619 06:49:29.419560 4568 ts_tablet_manager.cc:1939] T 856d6501fe6d47b19e55f42e167c5cc6 P 15422b3e22e24ffb8424459f175ba32a: Deleting consensus metadata
I20260619 06:49:29.420017 4257 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:29.421815 4063 catalog_manager.cc:4841] TS 15422b3e22e24ffb8424459f175ba32a (127.16.79.67:39449): tablet 856d6501fe6d47b19e55f42e167c5cc6 (table test-table [id=504ed2781bc645f1ba97984a19ffb7e0]) successfully deleted
I20260619 06:49:29.535717 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 4135
I20260619 06:49:29.554180 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 4266
W20260619 06:49:29.554234 4063 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.16.79.65:43225 (error 108)
I20260619 06:49:29.557235 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:29.555726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:43225 (local address 127.0.0.1:33454)
0619 06:49:29.556847 (+ 1121us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:29.556994 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:43225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":993,"thread_start_us":895,"threads_started":1}
W20260619 06:49:29.557561 4063 catalog_manager.cc:4584] TS 94194c6173074fd39f0f898368332177 (127.16.79.65:43225): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 856d6501fe6d47b19e55f42e167c5cc6: Network error: Client connection negotiation failed: client connection to 127.16.79.65:43225: connect: Connection refused (error 111)
I20260619 06:49:29.562476 4537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:29.555431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60058 (local address 127.16.79.66:38705)
0619 06:49:29.555628 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:29.555634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:29.555654 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:49:29.555722 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:29.555726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:29.555877 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:29.556024 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:29.556542 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:29.557517 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:29.559085 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:29.559612 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:29.559771 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:29.561029 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:29.561039 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:29.561054 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:29.561676 (+ 622us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:29.561733 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:29.561754 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:29.561875 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:29.562014 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 06:49:29.562120 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 06:49:29.563165 4388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:29.611294 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:29.610716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:43225 (local address 127.0.0.1:33466)
0619 06:49:29.611048 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:29.611146 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:43225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
I20260619 06:49:29.612677 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:29.612257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:43225 (local address 127.0.0.1:33482)
0619 06:49:29.612455 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:29.612526 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:43225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 06:49:29.673568 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:29.673056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:43225 (local address 127.0.0.1:33488)
0619 06:49:29.673303 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:29.673397 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:43225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260619 06:49:29.674772 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:29.674404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:43225 (local address 127.0.0.1:33494)
0619 06:49:29.674547 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:29.674618 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:43225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 06:49:29.677510 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 4266
I20260619 06:49:29.696569 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 4396
I20260619 06:49:29.703320 4540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:29.697267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46670 (local address 127.16.79.67:39449)
0619 06:49:29.697456 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:29.697464 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:29.697485 (+ 21us) server_negotiation.cc:407] Connection header received
0619 06:49:29.697698 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:29.697707 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:29.697799 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:29.697907 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:29.698307 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:29.699009 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:29.700333 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:29.700728 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:29.700867 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:29.701977 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:29.701985 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:29.701999 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:29.702552 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:29.702597 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:29.702621 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:29.702745 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:29.702877 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 06:49:29.702981 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 06:49:29.703987 4518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:29.755386 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:29.754840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:43225 (local address 127.0.0.1:33498)
0619 06:49:29.755090 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:29.755203 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:43225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260619 06:49:29.756546 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:29.756058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.65:43225 (local address 127.0.0.1:33514)
0619 06:49:29.756291 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 06:49:29.756376 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.65:43225: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260619 06:49:29.795378 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 4396
I20260619 06:49:29.811818 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 4045
I20260619 06:49:29.813047 4106 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:29.922778 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 4045
2026-06-19T06:49:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3148 ms)
[----------] 6 tests from EncryptionPolicyTest (22294 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:30 dist-test-slave-5w2x krb5kdc[4578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:30 dist-test-slave-5w2x krb5kdc[4578](info): set up 2 sockets
Jun 19 06:49:30 dist-test-slave-5w2x krb5kdc[4578](info): commencing operation
krb5kdc: starting...
W20260619 06:49:31.048712 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:31 dist-test-slave-5w2x krb5kdc[4578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:49:31.329923 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41539
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:31.674516 4591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:31.674887 4591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:31.675047 4591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:31.682914 4591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:31.683089 4591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:31.683182 4591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:31.683295 4591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:31.683365 4591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:31.693426 4591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41539
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:31.695101 4591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:31.696549 4591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:31.708631 4596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:31.708578 4597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:31.708655 4599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:31.709506 4591 server_base.cc:1034] running on GCE node
I20260619 06:49:31.710278 4591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:31.711720 4591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:31.712993 4591 hybrid_clock.cc:648] HybridClock initialized: now 1781851771712938 us; error 40 us; skew 500 ppm
I20260619 06:49:31.713459 4591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:31 dist-test-slave-5w2x krb5kdc[4578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851771, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:31.716863 4591 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:49:31.718578 4591 webserver.cc:460] Webserver started at http://127.0.0.1:35733/ using document root <none> and password file <none>
I20260619 06:49:31.719321 4591 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:31.719476 4591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:31.719794 4591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:31.722889 4591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "12eefcdfbd4f4e199347da103993e069"
format_stamp: "Formatted at 2026-06-19 06:49:31 on dist-test-slave-5w2x"
server_key: "8cba8b63f03716559babc4153c9d7ab1"
server_key_iv: "903a8b7531edf71f37df57f515f1038a"
server_key_version: "encryptionkey@0"
I20260619 06:49:31.723696 4591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "12eefcdfbd4f4e199347da103993e069"
format_stamp: "Formatted at 2026-06-19 06:49:31 on dist-test-slave-5w2x"
server_key: "8cba8b63f03716559babc4153c9d7ab1"
server_key_iv: "903a8b7531edf71f37df57f515f1038a"
server_key_version: "encryptionkey@0"
I20260619 06:49:31.729141 4591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 06:49:31.732754 4606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:31.734160 4591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260619 06:49:31.734349 4591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "12eefcdfbd4f4e199347da103993e069"
format_stamp: "Formatted at 2026-06-19 06:49:31 on dist-test-slave-5w2x"
server_key: "8cba8b63f03716559babc4153c9d7ab1"
server_key_iv: "903a8b7531edf71f37df57f515f1038a"
server_key_version: "encryptionkey@0"
I20260619 06:49:31.734549 4591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:31.754325 4591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:31.759176 4591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:31.759482 4591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:31.778306 4591 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41539
I20260619 06:49:31.778342 4657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41539 every 8 connection(s)
I20260619 06:49:31.779690 4591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:31.783411 4658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:31.785974 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 4591
I20260619 06:49:31.786209 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:31.786890 16701 external_mini_cluster.cc:1428] Setting key a690a149da1d3c7fb181ee3f16b7509b
I20260619 06:49:31.796527 4658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069: Bootstrap starting.
I20260619 06:49:31.800251 4658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:31.801280 4658 log.cc:824] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069: Log is configured to *not* fsync() on all Append() calls
Jun 19 06:49:31 dist-test-slave-5w2x krb5kdc[4578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 06:49:31.804675 4658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069: No bootstrap required, opened a new log
I20260619 06:49:31.811957 4661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:31.789035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48831 (local address 127.0.0.1:41539)
0619 06:49:31.789777 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:31.789793 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:31.789842 (+ 49us) server_negotiation.cc:407] Connection header received
0619 06:49:31.792300 (+ 2458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:31.792331 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:31.792891 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:31.793355 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:31.794677 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:31.796680 (+ 2003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:31.797992 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:31.799075 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:31.799268 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:31.803825 (+ 4557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:31.803837 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:31.803857 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:31.803901 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:31.807429 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:31.808351 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:31.808358 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:31.808365 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:31.808534 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:31.809052 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:31.809056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:31.809058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:31.809636 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:31.809986 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:31.810518 (+ 532us) server_negotiation.cc:299] Negotiation successful
0619 06:49:31.810875 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":291,"threads_started":1}
I20260619 06:49:31.812841 4658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12eefcdfbd4f4e199347da103993e069" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41539 } }
I20260619 06:49:31.813254 4658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:31.813411 4658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12eefcdfbd4f4e199347da103993e069, State: Initialized, Role: FOLLOWER
I20260619 06:49:31.814080 4658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12eefcdfbd4f4e199347da103993e069" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41539 } }
I20260619 06:49:31.814356 4658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:31.814522 4658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:31.814705 4658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:31.816248 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:49:31.817384 4658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12eefcdfbd4f4e199347da103993e069" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41539 } }
I20260619 06:49:31.817945 4658 leader_election.cc:302] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12eefcdfbd4f4e199347da103993e069; no voters:
I20260619 06:49:31.818565 4658 leader_election.cc:288] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:31.818714 4663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:31.820674 4658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:31.829530 4663 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [term 1 LEADER]: Becoming Leader. State: Replica: 12eefcdfbd4f4e199347da103993e069, State: Running, Role: LEADER
I20260619 06:49:31.830565 4663 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12eefcdfbd4f4e199347da103993e069" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41539 } }
I20260619 06:49:31.834066 4664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12eefcdfbd4f4e199347da103993e069" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12eefcdfbd4f4e199347da103993e069" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41539 } } }
I20260619 06:49:31.834424 4664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:31.834882 4666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12eefcdfbd4f4e199347da103993e069. Latest consensus state: current_term: 1 leader_uuid: "12eefcdfbd4f4e199347da103993e069" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12eefcdfbd4f4e199347da103993e069" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41539 } } }
I20260619 06:49:31.835201 4666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:31.835328 4682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:31.842700 4683 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 06:49:31.847635 4682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:31.858976 4682 catalog_manager.cc:1347] Generated new cluster ID: 163e558da0a74bb08682ad1559a9444b
I20260619 06:49:31.859099 4682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:31.859974 4661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:31.833476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45663 (local address 127.0.0.1:41539)
0619 06:49:31.833731 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:31.833740 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:31.842360 (+ 8620us) server_negotiation.cc:407] Connection header received
0619 06:49:31.842624 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:31.842634 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:31.842730 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:31.842866 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:31.845071 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:31.846405 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:31.848326 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:31.849074 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:31.849236 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:31.852662 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:31.852668 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:31.852672 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:31.852713 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:31.856005 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:31.857158 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:31.857165 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:31.857169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:31.857332 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:31.857926 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:31.857932 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:31.857937 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:31.858430 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:31.858748 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:31.859357 (+ 609us) server_negotiation.cc:299] Negotiation successful
0619 06:49:31.859575 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 06:49:31.869980 4682 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:31.871383 4682 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:31.886502 4682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12eefcdfbd4f4e199347da103993e069: Generated new TSK 0
I20260619 06:49:31.887452 4682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:31.903545 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 4591
I20260619 06:49:31.905310 4653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:32.001190 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 4591
W20260619 06:49:32.018630 4576 connection.cc:535] client connection to 127.0.0.1:41539 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41539): Connection reset by peer (error 104)
2026-06-19T06:49:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2072 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:32 dist-test-slave-5w2x krb5kdc[4690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:32 dist-test-slave-5w2x krb5kdc[4690](info): set up 2 sockets
Jun 19 06:49:32 dist-test-slave-5w2x krb5kdc[4690](info): commencing operation
krb5kdc: starting...
W20260619 06:49:33.124914 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:33 dist-test-slave-5w2x krb5kdc[4690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:49:33.385321 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39999
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:33.721939 4703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:33.722241 4703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:33.722404 4703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:33.729645 4703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:33.729759 4703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:33.729846 4703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:33.729951 4703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:33.730038 4703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:33.740080 4703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39999
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:33.741442 4703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:33.742751 4703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:33.755365 4709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:33.755388 4711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:33.755316 4708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:33.756189 4703 server_base.cc:1034] running on GCE node
I20260619 06:49:33.756932 4703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:33.758323 4703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:33.759531 4703 hybrid_clock.cc:648] HybridClock initialized: now 1781851773759472 us; error 49 us; skew 500 ppm
I20260619 06:49:33.759847 4703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:33 dist-test-slave-5w2x krb5kdc[4690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851773, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:33.763046 4703 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:49:33.764703 4703 webserver.cc:460] Webserver started at http://127.0.0.1:45907/ using document root <none> and password file <none>
I20260619 06:49:33.765404 4703 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:33.765530 4703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:33.765854 4703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:33.768874 4703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "5a8b07cc33774694a2f7b6ccdf33d2bd"
format_stamp: "Formatted at 2026-06-19 06:49:33 on dist-test-slave-5w2x"
server_key: "72190d55f779c1c87f71805e1731d400"
server_key_iv: "47c80adffe550438b941c6f548799f5a"
server_key_version: "encryptionkey@0"
I20260619 06:49:33.769618 4703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "5a8b07cc33774694a2f7b6ccdf33d2bd"
format_stamp: "Formatted at 2026-06-19 06:49:33 on dist-test-slave-5w2x"
server_key: "72190d55f779c1c87f71805e1731d400"
server_key_iv: "47c80adffe550438b941c6f548799f5a"
server_key_version: "encryptionkey@0"
I20260619 06:49:33.774739 4703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:49:33.778322 4718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:33.779794 4703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:49:33.780068 4703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "5a8b07cc33774694a2f7b6ccdf33d2bd"
format_stamp: "Formatted at 2026-06-19 06:49:33 on dist-test-slave-5w2x"
server_key: "72190d55f779c1c87f71805e1731d400"
server_key_iv: "47c80adffe550438b941c6f548799f5a"
server_key_version: "encryptionkey@0"
I20260619 06:49:33.780337 4703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:33.800537 4703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:33.804785 4703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:33.805092 4703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:33.823777 4703 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39999
I20260619 06:49:33.823812 4769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39999 every 8 connection(s)
I20260619 06:49:33.825239 4703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:33.828950 4770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:33.829947 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 4703
I20260619 06:49:33.830186 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:33.830888 16701 external_mini_cluster.cc:1428] Setting key 5833277fdd53ebe2555baa743d1bfe2a
I20260619 06:49:33.842844 4770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd: Bootstrap starting.
I20260619 06:49:33.844442 4773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:33.832733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48279 (local address 127.0.0.1:39999)
0619 06:49:33.833548 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:33.833559 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:33.833598 (+ 39us) server_negotiation.cc:407] Connection header received
0619 06:49:33.835350 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:33.835375 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:33.835762 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:33.836099 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:33.836707 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:33.837963 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:33.839879 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:33.840698 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:33.840854 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:33.842159 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:33.842185 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:33.842199 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:33.842723 (+ 524us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:33.842795 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:33.842819 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:33.842937 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:33.843266 (+ 329us) server_negotiation.cc:299] Negotiation successful
0619 06:49:33.843446 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":401,"threads_started":1}
I20260619 06:49:33.846065 4770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:33.847234 4770 log.cc:824] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:33.848218 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:49:33.850078 4770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd: No bootstrap required, opened a new log
I20260619 06:49:33.859457 4773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:33.852561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34827 (local address 127.0.0.1:39999)
0619 06:49:33.852741 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:33.852747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:33.853242 (+ 495us) server_negotiation.cc:407] Connection header received
0619 06:49:33.853397 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:33.853402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:33.853486 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:33.853589 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:33.854083 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:33.855315 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:33.856701 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:33.857483 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:33.857617 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:33.858627 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:33.858633 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:33.858637 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:33.858656 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:33.858716 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:33.858736 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:33.858851 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:33.859065 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 06:49:33.859126 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:49:33.859285 4770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8b07cc33774694a2f7b6ccdf33d2bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39999 } }
I20260619 06:49:33.859846 4770 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:33.859977 4770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a8b07cc33774694a2f7b6ccdf33d2bd, State: Initialized, Role: FOLLOWER
I20260619 06:49:33.860692 4770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8b07cc33774694a2f7b6ccdf33d2bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39999 } }
I20260619 06:49:33.861032 4770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:33.861167 4770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:33.861348 4770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:33.863490 4770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8b07cc33774694a2f7b6ccdf33d2bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39999 } }
I20260619 06:49:33.863991 4770 leader_election.cc:302] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a8b07cc33774694a2f7b6ccdf33d2bd; no voters:
I20260619 06:49:33.864616 4770 leader_election.cc:288] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:33.864743 4780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:33.866822 4770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:33.868453 4780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [term 1 LEADER]: Becoming Leader. State: Replica: 5a8b07cc33774694a2f7b6ccdf33d2bd, State: Running, Role: LEADER
I20260619 06:49:33.869114 4780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8b07cc33774694a2f7b6ccdf33d2bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39999 } }
I20260619 06:49:33.873490 4782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a8b07cc33774694a2f7b6ccdf33d2bd. Latest consensus state: current_term: 1 leader_uuid: "5a8b07cc33774694a2f7b6ccdf33d2bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8b07cc33774694a2f7b6ccdf33d2bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39999 } } }
I20260619 06:49:33.873871 4782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:33.873615 4781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a8b07cc33774694a2f7b6ccdf33d2bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a8b07cc33774694a2f7b6ccdf33d2bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39999 } } }
I20260619 06:49:33.873994 4781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd [sys.catalog]: This master's current role is: LEADER
W20260619 06:49:33.879251 4794 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:49:33.879364 4794 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:49:33.879514 4795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:33.881206 4795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:33.890082 4795 catalog_manager.cc:1347] Generated new cluster ID: a354e233b3a34e729a6d53e49bbc9d70
I20260619 06:49:33.890224 4795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:33.917418 4795 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:33.918881 4795 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:33.924515 4795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a8b07cc33774694a2f7b6ccdf33d2bd: Generated new TSK 0
I20260619 06:49:33.925398 4795 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:33.931213 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 4703
W20260619 06:49:33.932796 4765 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48279
W20260619 06:49:33.933576 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 4703: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:49:33.933744 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 4703
W20260619 06:49:33.947933 4688 connection.cc:535] client connection to 127.0.0.1:39999 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39999): Connection reset by peer (error 104)
2026-06-19T06:49:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1928 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:34 dist-test-slave-5w2x krb5kdc[4801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:34 dist-test-slave-5w2x krb5kdc[4801](info): set up 2 sockets
Jun 19 06:49:34 dist-test-slave-5w2x krb5kdc[4801](info): commencing operation
krb5kdc: starting...
W20260619 06:49:35.052510 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:35 dist-test-slave-5w2x krb5kdc[4801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:49:35.311278 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42553
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:35.644956 4814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:35.645300 4814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:35.645395 4814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:35.652565 4814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:35.652690 4814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:35.652735 4814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:35.652796 4814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:35.652833 4814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:35.662683 4814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42553
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:35.663980 4814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:35.665284 4814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:35.677565 4819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:35.677497 4820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:35.677600 4822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:35.678442 4814 server_base.cc:1034] running on GCE node
I20260619 06:49:35.679209 4814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:35.680720 4814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:35.681953 4814 hybrid_clock.cc:648] HybridClock initialized: now 1781851775681883 us; error 57 us; skew 500 ppm
I20260619 06:49:35.682313 4814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:35 dist-test-slave-5w2x krb5kdc[4801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:35.694415 4814 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:49:35.696333 4814 webserver.cc:460] Webserver started at http://127.0.0.1:42273/ using document root <none> and password file <none>
I20260619 06:49:35.697239 4814 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:35.697394 4814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:35.697765 4814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:35.701105 4814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "db82fdcec083475caf4c581d1dd1921f"
format_stamp: "Formatted at 2026-06-19 06:49:35 on dist-test-slave-5w2x"
server_key: "6c99ef0fbb2d0a7d8a34f05135188e30"
server_key_iv: "ddcbc4f1679c1c70976691efdc3f7852"
server_key_version: "encryptionkey@0"
I20260619 06:49:35.701993 4814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "db82fdcec083475caf4c581d1dd1921f"
format_stamp: "Formatted at 2026-06-19 06:49:35 on dist-test-slave-5w2x"
server_key: "6c99ef0fbb2d0a7d8a34f05135188e30"
server_key_iv: "ddcbc4f1679c1c70976691efdc3f7852"
server_key_version: "encryptionkey@0"
I20260619 06:49:35.707657 4814 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260619 06:49:35.711540 4829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:35.713001 4814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:49:35.713200 4814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "db82fdcec083475caf4c581d1dd1921f"
format_stamp: "Formatted at 2026-06-19 06:49:35 on dist-test-slave-5w2x"
server_key: "6c99ef0fbb2d0a7d8a34f05135188e30"
server_key_iv: "ddcbc4f1679c1c70976691efdc3f7852"
server_key_version: "encryptionkey@0"
I20260619 06:49:35.713419 4814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:35.719677 4814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:35.723851 4814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:35.724248 4814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:35.743381 4814 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42553
I20260619 06:49:35.743422 4880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42553 every 8 connection(s)
I20260619 06:49:35.744961 4814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:35.747875 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 4814
I20260619 06:49:35.748116 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:35.748625 16701 external_mini_cluster.cc:1428] Setting key 46b3c52591072057a01eda7b1f32a41a
I20260619 06:49:35.749089 4881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:35.756583 4884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:35.750419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59627 (local address 127.0.0.1:42553)
0619 06:49:35.751234 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:35.751245 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:35.751289 (+ 44us) server_negotiation.cc:407] Connection header received
0619 06:49:35.752913 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:35.752935 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:35.753303 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:35.753729 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:35.754795 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:35.754801 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:35.754814 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:35.754827 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:35.754905 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:35.754925 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:35.755173 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:35.755332 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 06:49:35.755516 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":358,"threads_started":1}
I20260619 06:49:35.760546 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:49:35.763070 4881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f: Bootstrap starting.
I20260619 06:49:35.766934 4881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:35.767572 4884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:35.765064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55351 (local address 127.0.0.1:42553)
0619 06:49:35.765271 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:35.765279 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:35.765629 (+ 350us) server_negotiation.cc:407] Connection header received
0619 06:49:35.765834 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:35.765841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:35.765892 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:35.766004 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:35.766951 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:35.766956 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:35.766968 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:35.766983 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:35.767021 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:35.767037 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:35.767162 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:35.767311 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 06:49:35.767359 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 06:49:35.768422 4881 log.cc:824] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:35.771306 4881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f: No bootstrap required, opened a new log
I20260619 06:49:35.779254 4881 raft_consensus.cc:348] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db82fdcec083475caf4c581d1dd1921f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42553 } }
I20260619 06:49:35.779690 4881 raft_consensus.cc:374] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:35.779768 4881 raft_consensus.cc:729] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db82fdcec083475caf4c581d1dd1921f, State: Initialized, Role: FOLLOWER
I20260619 06:49:35.780481 4881 consensus_queue.cc:260] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db82fdcec083475caf4c581d1dd1921f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42553 } }
I20260619 06:49:35.780779 4881 raft_consensus.cc:388] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:35.780956 4881 raft_consensus.cc:482] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:35.781138 4881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:35.783344 4881 raft_consensus.cc:504] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db82fdcec083475caf4c581d1dd1921f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42553 } }
I20260619 06:49:35.783821 4881 leader_election.cc:302] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db82fdcec083475caf4c581d1dd1921f; no voters:
I20260619 06:49:35.784514 4881 leader_election.cc:288] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:35.784682 4891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:35.785404 4891 raft_consensus.cc:686] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [term 1 LEADER]: Becoming Leader. State: Replica: db82fdcec083475caf4c581d1dd1921f, State: Running, Role: LEADER
I20260619 06:49:35.786008 4891 consensus_queue.cc:237] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db82fdcec083475caf4c581d1dd1921f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42553 } }
I20260619 06:49:35.786746 4881 sys_catalog.cc:564] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:35.788648 4892 sys_catalog.cc:455] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db82fdcec083475caf4c581d1dd1921f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db82fdcec083475caf4c581d1dd1921f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42553 } } }
I20260619 06:49:35.789124 4892 sys_catalog.cc:458] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:35.789187 4893 sys_catalog.cc:455] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [sys.catalog]: SysCatalogTable state changed. Reason: New leader db82fdcec083475caf4c581d1dd1921f. Latest consensus state: current_term: 1 leader_uuid: "db82fdcec083475caf4c581d1dd1921f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db82fdcec083475caf4c581d1dd1921f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42553 } } }
I20260619 06:49:35.789475 4893 sys_catalog.cc:458] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:35.791725 4897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:35.797500 4897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:35.806949 4897 catalog_manager.cc:1347] Generated new cluster ID: 59c7a4b7e5a74191b7ab419844f35833
I20260619 06:49:35.807083 4897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:35.824633 4897 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:35.825124 4897 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:35.832324 4897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db82fdcec083475caf4c581d1dd1921f: Generated new TSK 0
I20260619 06:49:35.833235 4897 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:35.842245 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 4814
W20260619 06:49:35.843684 4876 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59627
W20260619 06:49:35.844463 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 4814: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:49:35.844619 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 4814
2026-06-19T06:49:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1913 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:35 dist-test-slave-5w2x krb5kdc[4912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:35 dist-test-slave-5w2x krb5kdc[4912](info): set up 2 sockets
Jun 19 06:49:35 dist-test-slave-5w2x krb5kdc[4912](info): commencing operation
krb5kdc: starting...
W20260619 06:49:36.940658 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.005s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:37 dist-test-slave-5w2x krb5kdc[4912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:49:37.184971 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:37.521837 4925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:37.522145 4925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:37.522240 4925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:37.529672 4925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:37.529789 4925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:37.529835 4925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:37.529896 4925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:37.529942 4925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:37.540433 4925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:37.541759 4925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:37.543066 4925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:37.555177 4930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:37.555155 4931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:37.555115 4933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:37.556252 4925 server_base.cc:1034] running on GCE node
I20260619 06:49:37.557014 4925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:37.558529 4925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:37.559738 4925 hybrid_clock.cc:648] HybridClock initialized: now 1781851777559697 us; error 31 us; skew 500 ppm
I20260619 06:49:37.560055 4925 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:37 dist-test-slave-5w2x krb5kdc[4912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851777, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:37.563268 4925 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:49:37.564828 4925 webserver.cc:460] Webserver started at http://127.0.0.1:33011/ using document root <none> and password file <none>
I20260619 06:49:37.565492 4925 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:37.565593 4925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:37.565848 4925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:37.568917 4925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "a012a829442e4d98891ec2de7df1bbce"
format_stamp: "Formatted at 2026-06-19 06:49:37 on dist-test-slave-5w2x"
server_key: "861f0ec5be18982103ebc515bd911fb9"
server_key_iv: "b25fc32960489742ad14ae7a661db8fa"
server_key_version: "encryptionkey@0"
I20260619 06:49:37.569624 4925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "a012a829442e4d98891ec2de7df1bbce"
format_stamp: "Formatted at 2026-06-19 06:49:37 on dist-test-slave-5w2x"
server_key: "861f0ec5be18982103ebc515bd911fb9"
server_key_iv: "b25fc32960489742ad14ae7a661db8fa"
server_key_version: "encryptionkey@0"
I20260619 06:49:37.574822 4925 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:49:37.578222 4940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:37.579561 4925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:49:37.579704 4925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "a012a829442e4d98891ec2de7df1bbce"
format_stamp: "Formatted at 2026-06-19 06:49:37 on dist-test-slave-5w2x"
server_key: "861f0ec5be18982103ebc515bd911fb9"
server_key_iv: "b25fc32960489742ad14ae7a661db8fa"
server_key_version: "encryptionkey@0"
I20260619 06:49:37.579865 4925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:37.600045 4925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:37.604637 4925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:37.604902 4925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:37.623695 4925 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44717
I20260619 06:49:37.623741 4991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44717 every 8 connection(s)
I20260619 06:49:37.625219 4925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:37.629181 4992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:37.629525 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 4925
I20260619 06:49:37.629700 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:37.630172 16701 external_mini_cluster.cc:1428] Setting key ac3524ef9432b20b29c1ef3f97bb3593
I20260619 06:49:37.643443 4992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce: Bootstrap starting.
Jun 19 06:49:37 dist-test-slave-5w2x krb5kdc[4912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 06:49:37.647197 4992 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:37.648373 4992 log.cc:824] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:37.651821 4992 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce: No bootstrap required, opened a new log
I20260619 06:49:37.655782 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:37.631991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54495 (local address 127.0.0.1:44717)
0619 06:49:37.632922 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:37.632934 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:37.632974 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:49:37.635352 (+ 2378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:37.635378 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:37.635817 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:37.636236 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:37.638155 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:37.639490 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:37.641348 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:37.642012 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:37.642187 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:37.647018 (+ 4831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:37.647029 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:37.647046 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:37.647086 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:37.650892 (+ 3806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:37.651604 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:37.651612 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:37.651620 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:37.651836 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:37.652411 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:37.652417 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:37.652420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:37.653151 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:37.653460 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:37.653894 (+ 434us) server_negotiation.cc:299] Negotiation successful
0619 06:49:37.654373 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":473,"threads_started":1}
I20260619 06:49:37.659672 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:49:37.661453 4992 raft_consensus.cc:348] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a012a829442e4d98891ec2de7df1bbce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44717 } }
I20260619 06:49:37.661888 4992 raft_consensus.cc:374] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:37.661988 4992 raft_consensus.cc:729] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a012a829442e4d98891ec2de7df1bbce, State: Initialized, Role: FOLLOWER
I20260619 06:49:37.662647 4992 consensus_queue.cc:260] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a012a829442e4d98891ec2de7df1bbce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44717 } }
I20260619 06:49:37.662945 4992 raft_consensus.cc:388] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:37.663118 4992 raft_consensus.cc:482] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:37.663291 4992 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:37.666659 4992 raft_consensus.cc:504] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a012a829442e4d98891ec2de7df1bbce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44717 } }
I20260619 06:49:37.667287 4992 leader_election.cc:302] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a012a829442e4d98891ec2de7df1bbce; no voters:
I20260619 06:49:37.667984 4992 leader_election.cc:288] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:37.668048 5002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:37.669854 5002 raft_consensus.cc:686] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [term 1 LEADER]: Becoming Leader. State: Replica: a012a829442e4d98891ec2de7df1bbce, State: Running, Role: LEADER
I20260619 06:49:37.670495 5002 consensus_queue.cc:237] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a012a829442e4d98891ec2de7df1bbce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44717 } }
I20260619 06:49:37.670627 4992 sys_catalog.cc:564] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:37.677798 5004 sys_catalog.cc:455] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [sys.catalog]: SysCatalogTable state changed. Reason: New leader a012a829442e4d98891ec2de7df1bbce. Latest consensus state: current_term: 1 leader_uuid: "a012a829442e4d98891ec2de7df1bbce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a012a829442e4d98891ec2de7df1bbce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44717 } } }
I20260619 06:49:37.678274 5004 sys_catalog.cc:458] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:37.678267 5003 sys_catalog.cc:455] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a012a829442e4d98891ec2de7df1bbce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a012a829442e4d98891ec2de7df1bbce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44717 } } }
I20260619 06:49:37.678579 5003 sys_catalog.cc:458] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:37.682582 4995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:37.665247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56077 (local address 127.0.0.1:44717)
0619 06:49:37.665445 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:37.665451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:37.665962 (+ 511us) server_negotiation.cc:407] Connection header received
0619 06:49:37.666126 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:37.666130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:37.666200 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:37.666274 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:37.668343 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:37.669463 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:37.671026 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:37.671507 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:37.671647 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:37.674326 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:37.674332 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:37.674335 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:37.674365 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:37.677925 (+ 3560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:37.680022 (+ 2097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:37.680029 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:37.680034 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:37.680409 (+ 375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:37.680858 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:37.680862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:37.680864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:37.681221 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:37.681496 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:37.681827 (+ 331us) server_negotiation.cc:299] Negotiation successful
0619 06:49:37.682049 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":65}
I20260619 06:49:37.688109 5008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:37.693488 5008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:37.701823 5008 catalog_manager.cc:1347] Generated new cluster ID: d231cd274e384f1f85c44478db576037
I20260619 06:49:37.701941 5008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:37.713865 5008 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:37.715801 5008 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:37.723342 5008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a012a829442e4d98891ec2de7df1bbce: Generated new TSK 0
I20260619 06:49:37.724437 5008 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:37.732055 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 4925
I20260619 06:49:37.733737 4987 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:37.824241 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 4925
2026-06-19T06:49:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1979 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:37 dist-test-slave-5w2x krb5kdc[5024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:37 dist-test-slave-5w2x krb5kdc[5024](info): set up 2 sockets
Jun 19 06:49:37 dist-test-slave-5w2x krb5kdc[5024](info): commencing operation
krb5kdc: starting...
W20260619 06:49:38.920656 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.005s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:39 dist-test-slave-5w2x krb5kdc[5024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:49:39.174985 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42007
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42007
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:39.517374 5037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:39.517680 5037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:39.517772 5037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:39.525561 5037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:39.525698 5037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:39.525746 5037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:39.525811 5037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:39.525857 5037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:39.536412 5037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:39.537744 5037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:39.539099 5037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:39.550734 5043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:39.550805 5045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:39.551115 5042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:39.552212 5037 server_base.cc:1034] running on GCE node
I20260619 06:49:39.553066 5037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:39.554512 5037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:39.555758 5037 hybrid_clock.cc:648] HybridClock initialized: now 1781851779555726 us; error 56 us; skew 500 ppm
I20260619 06:49:39.556125 5037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:39 dist-test-slave-5w2x krb5kdc[5024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851779, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:39.559542 5037 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:49:39.561161 5037 webserver.cc:460] Webserver started at http://127.0.0.1:39455/ using document root <none> and password file <none>
I20260619 06:49:39.561892 5037 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:39.562050 5037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:39.562364 5037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:39.565683 5037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "0e1663680d5b4f0681874329fc21b5ff"
format_stamp: "Formatted at 2026-06-19 06:49:39 on dist-test-slave-5w2x"
server_key: "3f99ec4429f66043c5e16d62ada34a0e"
server_key_iv: "bcb31f2399066e02ef23b76baacc1d5b"
server_key_version: "encryptionkey@0"
I20260619 06:49:39.566468 5037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "0e1663680d5b4f0681874329fc21b5ff"
format_stamp: "Formatted at 2026-06-19 06:49:39 on dist-test-slave-5w2x"
server_key: "3f99ec4429f66043c5e16d62ada34a0e"
server_key_iv: "bcb31f2399066e02ef23b76baacc1d5b"
server_key_version: "encryptionkey@0"
I20260619 06:49:39.571724 5037 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:49:39.575305 5052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:39.576702 5037 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:49:39.576889 5037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "0e1663680d5b4f0681874329fc21b5ff"
format_stamp: "Formatted at 2026-06-19 06:49:39 on dist-test-slave-5w2x"
server_key: "3f99ec4429f66043c5e16d62ada34a0e"
server_key_iv: "bcb31f2399066e02ef23b76baacc1d5b"
server_key_version: "encryptionkey@0"
I20260619 06:49:39.577093 5037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:39.597666 5037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:39.602216 5037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:39.602521 5037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:39.621330 5037 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42007
I20260619 06:49:39.621351 5103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42007 every 8 connection(s)
I20260619 06:49:39.622689 5037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:39.626410 5104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:39.630646 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 5037
I20260619 06:49:39.630838 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:39.631268 16701 external_mini_cluster.cc:1428] Setting key 15b3c66e03dc4a69efcb474887896024
I20260619 06:49:39.637485 5104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff: Bootstrap starting.
I20260619 06:49:39.641189 5104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:39.642473 5104 log.cc:824] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:39.645602 5107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:39.633011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43185 (local address 127.0.0.1:42007)
0619 06:49:39.633795 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:39.633806 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:39.633847 (+ 41us) server_negotiation.cc:407] Connection header received
0619 06:49:39.635983 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:39.636006 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:39.636492 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:39.636808 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:39.637418 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:39.638751 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:39.640749 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:39.641601 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:39.641746 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:39.643028 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:39.643041 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:39.643062 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:39.643928 (+ 866us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:39.644010 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:39.644036 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:39.644191 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:39.644427 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 06:49:39.644677 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":363,"threads_started":1}
I20260619 06:49:39.645989 5104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff: No bootstrap required, opened a new log
I20260619 06:49:39.649477 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:49:39.654006 5104 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1663680d5b4f0681874329fc21b5ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } }
I20260619 06:49:39.654614 5104 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:39.654744 5104 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1663680d5b4f0681874329fc21b5ff, State: Initialized, Role: FOLLOWER
I20260619 06:49:39.655403 5104 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1663680d5b4f0681874329fc21b5ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } }
I20260619 06:49:39.655655 5104 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:39.655830 5104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:39.656032 5104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:39.658418 5104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1663680d5b4f0681874329fc21b5ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } }
I20260619 06:49:39.659044 5104 leader_election.cc:302] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e1663680d5b4f0681874329fc21b5ff; no voters:
I20260619 06:49:39.659723 5104 leader_election.cc:288] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:39.659849 5114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:39.660915 5107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:39.653970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38389 (local address 127.0.0.1:42007)
0619 06:49:39.654168 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:39.654174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:39.654761 (+ 587us) server_negotiation.cc:407] Connection header received
0619 06:49:39.655007 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:39.655011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:39.655076 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:39.655200 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:39.655775 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:39.656639 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:39.658026 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:39.658487 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:39.658624 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:39.659984 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:39.659993 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:39.659999 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:39.660028 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:39.660088 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:39.660128 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:39.660237 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:39.660494 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 06:49:39.660607 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 06:49:39.661805 5114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [term 1 LEADER]: Becoming Leader. State: Replica: 0e1663680d5b4f0681874329fc21b5ff, State: Running, Role: LEADER
I20260619 06:49:39.662215 5104 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:39.662529 5114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1663680d5b4f0681874329fc21b5ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } }
I20260619 06:49:39.665722 5115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e1663680d5b4f0681874329fc21b5ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1663680d5b4f0681874329fc21b5ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } } }
I20260619 06:49:39.665750 5116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e1663680d5b4f0681874329fc21b5ff. Latest consensus state: current_term: 1 leader_uuid: "0e1663680d5b4f0681874329fc21b5ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1663680d5b4f0681874329fc21b5ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42007 } } }
I20260619 06:49:39.666200 5115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:39.666242 5116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:39.667153 5123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:39.672905 5123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:39.683884 5123 catalog_manager.cc:1347] Generated new cluster ID: f60e8370ada14a5aae65e1b714e3ac07
I20260619 06:49:39.684046 5123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:39.708504 5123 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:39.710592 5123 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:39.725363 5123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e1663680d5b4f0681874329fc21b5ff: Generated new TSK 0
I20260619 06:49:39.726308 5123 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:39.734678 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 5037
W20260619 06:49:39.735862 5099 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43185
W20260619 06:49:39.736599 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 5037: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:49:39.736747 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 5037
2026-06-19T06:49:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1912 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:39 dist-test-slave-5w2x krb5kdc[5135](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:39 dist-test-slave-5w2x krb5kdc[5135](info): set up 2 sockets
Jun 19 06:49:39 dist-test-slave-5w2x krb5kdc[5135](info): commencing operation
krb5kdc: starting...
W20260619 06:49:40.844595 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.009s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:41 dist-test-slave-5w2x krb5kdc[5135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:49:41.095021 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32931
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:41.446959 5148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:41.447293 5148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:41.447450 5148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:41.454842 5148 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:41.454993 5148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:41.455102 5148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:41.455212 5148 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:41.455302 5148 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:41.465317 5148 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:41.466621 5148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:41.467976 5148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:41.480783 5154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:41.480855 5153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:41.480881 5156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:41.481367 5148 server_base.cc:1034] running on GCE node
I20260619 06:49:41.482219 5148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:41.483824 5148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:41.485086 5148 hybrid_clock.cc:648] HybridClock initialized: now 1781851781485029 us; error 40 us; skew 500 ppm
I20260619 06:49:41.485471 5148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:41 dist-test-slave-5w2x krb5kdc[5135](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851781, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:41.488771 5148 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:49:41.490236 5148 webserver.cc:460] Webserver started at http://127.0.0.1:37901/ using document root <none> and password file <none>
I20260619 06:49:41.491099 5148 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:41.491291 5148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:41.491629 5148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:41.494724 5148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "5059447d170141068942ba3cce281ef0"
format_stamp: "Formatted at 2026-06-19 06:49:41 on dist-test-slave-5w2x"
server_key: "0d2df06fd246af9dc40b6d36eca5df30"
server_key_iv: "a61e974183ebf4d305a9f529b2297d23"
server_key_version: "encryptionkey@0"
I20260619 06:49:41.495522 5148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "5059447d170141068942ba3cce281ef0"
format_stamp: "Formatted at 2026-06-19 06:49:41 on dist-test-slave-5w2x"
server_key: "0d2df06fd246af9dc40b6d36eca5df30"
server_key_iv: "a61e974183ebf4d305a9f529b2297d23"
server_key_version: "encryptionkey@0"
I20260619 06:49:41.500926 5148 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260619 06:49:41.504478 5163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:41.505927 5148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:49:41.506130 5148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "5059447d170141068942ba3cce281ef0"
format_stamp: "Formatted at 2026-06-19 06:49:41 on dist-test-slave-5w2x"
server_key: "0d2df06fd246af9dc40b6d36eca5df30"
server_key_iv: "a61e974183ebf4d305a9f529b2297d23"
server_key_version: "encryptionkey@0"
I20260619 06:49:41.506345 5148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:41.512347 5148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:41.516950 5148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:41.517266 5148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:41.536283 5148 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32931
I20260619 06:49:41.536319 5214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32931 every 8 connection(s)
I20260619 06:49:41.537782 5148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:41.539973 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 5148
I20260619 06:49:41.540211 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:41.540665 16701 external_mini_cluster.cc:1428] Setting key 2707da45f86c85b7ee21471cc68ff51a
I20260619 06:49:41.542194 5215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:41.549624 5218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:41.542579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33851 (local address 127.0.0.1:32931)
0619 06:49:41.543404 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:41.543423 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:41.543474 (+ 51us) server_negotiation.cc:407] Connection header received
0619 06:49:41.545500 (+ 2026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:41.545529 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:41.546051 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:41.546518 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:41.547549 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:41.547562 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:41.547582 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:41.547599 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:41.547693 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:41.547719 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:41.547958 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:41.548119 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 06:49:41.548341 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":338,"threads_started":1}
I20260619 06:49:41.553494 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:49:41.557420 5215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0: Bootstrap starting.
I20260619 06:49:41.561582 5218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:41.559050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59285 (local address 127.0.0.1:32931)
0619 06:49:41.559263 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:41.559272 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:41.559560 (+ 288us) server_negotiation.cc:407] Connection header received
0619 06:49:41.559742 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:41.559747 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:41.559810 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:41.559934 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:41.560886 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:41.560891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:41.560896 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:41.560913 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:41.560957 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:41.560978 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:41.561103 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:41.561179 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 06:49:41.561236 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 06:49:41.561549 5215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:41.562753 5215 log.cc:824] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:41.565589 5215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0: No bootstrap required, opened a new log
I20260619 06:49:41.573498 5215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5059447d170141068942ba3cce281ef0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32931 } }
I20260619 06:49:41.573901 5215 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:41.574016 5215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5059447d170141068942ba3cce281ef0, State: Initialized, Role: FOLLOWER
I20260619 06:49:41.574739 5215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5059447d170141068942ba3cce281ef0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32931 } }
I20260619 06:49:41.574958 5215 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:41.575073 5215 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:41.575196 5215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:41.577447 5215 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5059447d170141068942ba3cce281ef0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32931 } }
I20260619 06:49:41.577900 5215 leader_election.cc:302] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5059447d170141068942ba3cce281ef0; no voters:
I20260619 06:49:41.578887 5215 leader_election.cc:288] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:41.579013 5225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:41.579772 5225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [term 1 LEADER]: Becoming Leader. State: Replica: 5059447d170141068942ba3cce281ef0, State: Running, Role: LEADER
I20260619 06:49:41.580478 5225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5059447d170141068942ba3cce281ef0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32931 } }
I20260619 06:49:41.581442 5215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:41.583635 5227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5059447d170141068942ba3cce281ef0. Latest consensus state: current_term: 1 leader_uuid: "5059447d170141068942ba3cce281ef0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5059447d170141068942ba3cce281ef0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32931 } } }
I20260619 06:49:41.584045 5227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:41.586112 5226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5059447d170141068942ba3cce281ef0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5059447d170141068942ba3cce281ef0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32931 } } }
I20260619 06:49:41.586447 5231 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:41.586802 5226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:41.593163 5231 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:41.603729 5231 catalog_manager.cc:1347] Generated new cluster ID: 9018c16974bf4b03b1a43f51c1c5d145
I20260619 06:49:41.603864 5231 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:41.618661 5231 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:41.619220 5231 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:41.631393 5231 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5059447d170141068942ba3cce281ef0: Generated new TSK 0
I20260619 06:49:41.632396 5231 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:41.647081 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 5148
W20260619 06:49:41.648393 5210 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33851
W20260619 06:49:41.649179 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 5148: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:49:41.649303 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 5148
2026-06-19T06:49:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1912 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:41 dist-test-slave-5w2x krb5kdc[5246](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:41 dist-test-slave-5w2x krb5kdc[5246](info): set up 2 sockets
Jun 19 06:49:41 dist-test-slave-5w2x krb5kdc[5246](info): commencing operation
krb5kdc: starting...
W20260619 06:49:42.760854 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:42 dist-test-slave-5w2x krb5kdc[5246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:49:43.017771 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33589
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:43.353742 5259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:43.354123 5259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:43.354295 5259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:43.361724 5259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:43.361889 5259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:43.361995 5259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:43.362105 5259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:43.362188 5259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:43.373800 5259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:43.375182 5259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:43.376585 5259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:43.389429 5264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:43.389446 5267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:43.389592 5265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:43.391708 5259 server_base.cc:1034] running on GCE node
I20260619 06:49:43.392679 5259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:43.394197 5259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:43.395428 5259 hybrid_clock.cc:648] HybridClock initialized: now 1781851783395376 us; error 52 us; skew 500 ppm
I20260619 06:49:43.395795 5259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:43 dist-test-slave-5w2x krb5kdc[5246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851783, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:43.399084 5259 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:49:43.400709 5259 webserver.cc:460] Webserver started at http://127.0.0.1:39641/ using document root <none> and password file <none>
I20260619 06:49:43.401496 5259 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:43.401605 5259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:43.401872 5259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:43.404978 5259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "7f5be3061eee428a9771e543d9802ccf"
format_stamp: "Formatted at 2026-06-19 06:49:43 on dist-test-slave-5w2x"
server_key: "0ae075f5770c114df5f2ae7309250fe4"
server_key_iv: "559aafca0e12a96a092c6139bf7e8ffe"
server_key_version: "encryptionkey@0"
I20260619 06:49:43.405769 5259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "7f5be3061eee428a9771e543d9802ccf"
format_stamp: "Formatted at 2026-06-19 06:49:43 on dist-test-slave-5w2x"
server_key: "0ae075f5770c114df5f2ae7309250fe4"
server_key_iv: "559aafca0e12a96a092c6139bf7e8ffe"
server_key_version: "encryptionkey@0"
I20260619 06:49:43.411051 5259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:49:43.414685 5274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:43.416092 5259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:49:43.416352 5259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "7f5be3061eee428a9771e543d9802ccf"
format_stamp: "Formatted at 2026-06-19 06:49:43 on dist-test-slave-5w2x"
server_key: "0ae075f5770c114df5f2ae7309250fe4"
server_key_iv: "559aafca0e12a96a092c6139bf7e8ffe"
server_key_version: "encryptionkey@0"
I20260619 06:49:43.416535 5259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:43.428038 5259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:43.432369 5259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:43.432602 5259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:43.451249 5259 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33589
I20260619 06:49:43.451272 5325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33589 every 8 connection(s)
I20260619 06:49:43.452703 5259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:43.456512 5326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:43.462864 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 5259
I20260619 06:49:43.463073 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:43.463621 16701 external_mini_cluster.cc:1428] Setting key 20ca5fdf5d263b67dfd88459230f25ce
I20260619 06:49:43.467852 5326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf: Bootstrap starting.
I20260619 06:49:43.471773 5326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:43.472908 5326 log.cc:824] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:43.476015 5326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf: No bootstrap required, opened a new log
Jun 19 06:49:43 dist-test-slave-5w2x krb5kdc[5246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 06:49:43.484555 5326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5be3061eee428a9771e543d9802ccf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33589 } }
I20260619 06:49:43.484958 5326 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:43.485090 5326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f5be3061eee428a9771e543d9802ccf, State: Initialized, Role: FOLLOWER
I20260619 06:49:43.485713 5326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5be3061eee428a9771e543d9802ccf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33589 } }
I20260619 06:49:43.485960 5326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:43.486110 5326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:43.486302 5326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:43.487897 5329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:43.465938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:44011 (local address 127.0.0.1:33589)
0619 06:49:43.466763 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:43.466776 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:43.466817 (+ 41us) server_negotiation.cc:407] Connection header received
0619 06:49:43.468687 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:43.468711 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:43.469118 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:43.469469 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:43.471419 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:43.472876 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:43.474483 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:43.475144 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:43.475270 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:43.480100 (+ 4830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:43.480108 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:43.480125 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:43.480229 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:43.483384 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:43.484234 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:43.484241 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:43.484248 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:43.484413 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:43.484889 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:43.484893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:43.484895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:43.485615 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:43.485948 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:43.486442 (+ 494us) server_negotiation.cc:299] Negotiation successful
0619 06:49:43.486785 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":394,"threads_started":1}
I20260619 06:49:43.489039 5326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5be3061eee428a9771e543d9802ccf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33589 } }
I20260619 06:49:43.489557 5326 leader_election.cc:302] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f5be3061eee428a9771e543d9802ccf; no voters:
I20260619 06:49:43.490280 5326 leader_election.cc:288] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:43.490477 5331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:43.491600 5331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [term 1 LEADER]: Becoming Leader. State: Replica: 7f5be3061eee428a9771e543d9802ccf, State: Running, Role: LEADER
I20260619 06:49:43.492225 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:49:43.492255 5331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5be3061eee428a9771e543d9802ccf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33589 } }
I20260619 06:49:43.492936 5326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:43.495378 5332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f5be3061eee428a9771e543d9802ccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5be3061eee428a9771e543d9802ccf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33589 } } }
I20260619 06:49:43.495350 5333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f5be3061eee428a9771e543d9802ccf. Latest consensus state: current_term: 1 leader_uuid: "7f5be3061eee428a9771e543d9802ccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f5be3061eee428a9771e543d9802ccf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33589 } } }
I20260619 06:49:43.495841 5332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:43.495841 5333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:43.501957 5337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:43.509930 5337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:43.525894 5337 catalog_manager.cc:1347] Generated new cluster ID: 073dca1eab2d4370bbf80a7b5724de2f
I20260619 06:49:43.526343 5337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:43.530473 5329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:43.515193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:57873 (local address 127.0.0.1:33589)
0619 06:49:43.515388 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:43.515399 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:43.515418 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:49:43.515477 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:43.515483 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:43.515554 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:43.515667 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:43.517677 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:43.518890 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:43.520774 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:43.521468 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:43.521602 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:43.523978 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:43.523985 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:43.523990 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:43.524028 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:43.527457 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:43.528271 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:43.528277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:43.528281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:43.528403 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:43.528858 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:43.528863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:43.528866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:43.529181 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:43.529482 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:43.529898 (+ 416us) server_negotiation.cc:299] Negotiation successful
0619 06:49:43.530087 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 06:49:43.546823 5337 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:43.548189 5337 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:43.553946 5337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f5be3061eee428a9771e543d9802ccf: Generated new TSK 0
I20260619 06:49:43.554774 5337 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:43.561491 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 5259
I20260619 06:49:43.562705 5321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:43.653083 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 5259
W20260619 06:49:43.670018 5244 connection.cc:535] client connection to 127.0.0.1:33589 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33589): Connection reset by peer (error 104)
2026-06-19T06:49:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2003 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:43 dist-test-slave-5w2x krb5kdc[5358](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:43 dist-test-slave-5w2x krb5kdc[5358](info): set up 2 sockets
Jun 19 06:49:43 dist-test-slave-5w2x krb5kdc[5358](info): commencing operation
krb5kdc: starting...
W20260619 06:49:44.752637 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:44 dist-test-slave-5w2x krb5kdc[5358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:49:45.003300 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39323
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:45.355476 5371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:45.355844 5371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:45.356019 5371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:45.363620 5371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:45.363790 5371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:45.363893 5371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:45.364001 5371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:45.364089 5371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:45.374660 5371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:45.376459 5371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:45.378124 5371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:45.390574 5376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:45.390537 5379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:45.390586 5377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:45.391471 5371 server_base.cc:1034] running on GCE node
I20260619 06:49:45.392381 5371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:45.393977 5371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:45.395252 5371 hybrid_clock.cc:648] HybridClock initialized: now 1781851785395199 us; error 44 us; skew 500 ppm
I20260619 06:49:45.395653 5371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:45 dist-test-slave-5w2x krb5kdc[5358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:45.399003 5371 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:49:45.400785 5371 webserver.cc:460] Webserver started at http://127.0.0.1:45421/ using document root <none> and password file <none>
I20260619 06:49:45.401507 5371 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:45.401657 5371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:45.401962 5371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:45.405131 5371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "c2189c070db54489bc399c8063d50497"
format_stamp: "Formatted at 2026-06-19 06:49:45 on dist-test-slave-5w2x"
server_key: "be9d529ea3671fd1bb36c7f796e0bcfb"
server_key_iv: "0f546f2086a80130eb6d0d7865db8053"
server_key_version: "encryptionkey@0"
I20260619 06:49:45.405910 5371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "c2189c070db54489bc399c8063d50497"
format_stamp: "Formatted at 2026-06-19 06:49:45 on dist-test-slave-5w2x"
server_key: "be9d529ea3671fd1bb36c7f796e0bcfb"
server_key_iv: "0f546f2086a80130eb6d0d7865db8053"
server_key_version: "encryptionkey@0"
I20260619 06:49:45.411357 5371 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:49:45.414999 5386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:45.416466 5371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:49:45.416656 5371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "c2189c070db54489bc399c8063d50497"
format_stamp: "Formatted at 2026-06-19 06:49:45 on dist-test-slave-5w2x"
server_key: "be9d529ea3671fd1bb36c7f796e0bcfb"
server_key_iv: "0f546f2086a80130eb6d0d7865db8053"
server_key_version: "encryptionkey@0"
I20260619 06:49:45.416858 5371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:45.429278 5371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:45.433518 5371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:45.433820 5371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:45.452935 5371 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39323
I20260619 06:49:45.452955 5437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39323 every 8 connection(s)
I20260619 06:49:45.454401 5371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:45.458324 5438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:45.458427 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 5371
I20260619 06:49:45.458602 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:45.459197 16701 external_mini_cluster.cc:1428] Setting key 94b778b4894d35fb911cedddbcca96d1
I20260619 06:49:45.473609 5438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497: Bootstrap starting.
I20260619 06:49:45.473793 5441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:45.461131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:60355 (local address 127.0.0.1:39323)
0619 06:49:45.462030 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:45.462042 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:45.462086 (+ 44us) server_negotiation.cc:407] Connection header received
0619 06:49:45.463877 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:45.463903 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:45.464444 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:45.464831 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:45.465564 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:45.466972 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:45.468935 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:45.469713 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:45.469883 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:45.471285 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:45.471293 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:45.471309 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:45.471903 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:45.471993 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:45.472020 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:45.472219 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:45.472467 (+ 248us) server_negotiation.cc:299] Negotiation successful
0619 06:49:45.472704 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":451,"threads_started":1}
I20260619 06:49:45.477080 5438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:45.478281 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:49:45.478238 5438 log.cc:824] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:45.481812 5438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497: No bootstrap required, opened a new log
I20260619 06:49:45.490523 5441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:45.483495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:36751 (local address 127.0.0.1:39323)
0619 06:49:45.483709 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:45.483716 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:45.484083 (+ 367us) server_negotiation.cc:407] Connection header received
0619 06:49:45.484317 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:45.484323 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:45.484414 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:45.484527 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:45.485023 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:45.486248 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:45.487625 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:45.488361 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:45.488497 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:45.489697 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:45.489705 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:45.489710 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:45.489732 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:45.489796 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:45.489820 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:45.489941 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:45.490111 (+ 170us) server_negotiation.cc:299] Negotiation successful
0619 06:49:45.490171 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 06:49:45.491823 5438 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2189c070db54489bc399c8063d50497" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } }
I20260619 06:49:45.492324 5438 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:45.492426 5438 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2189c070db54489bc399c8063d50497, State: Initialized, Role: FOLLOWER
I20260619 06:49:45.493028 5438 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2189c070db54489bc399c8063d50497" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } }
I20260619 06:49:45.493325 5438 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:45.493487 5438 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:45.493654 5438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:45.495848 5438 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2189c070db54489bc399c8063d50497" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } }
I20260619 06:49:45.496410 5438 leader_election.cc:302] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2189c070db54489bc399c8063d50497; no voters:
I20260619 06:49:45.497073 5438 leader_election.cc:288] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:45.497217 5448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:45.497949 5448 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [term 1 LEADER]: Becoming Leader. State: Replica: c2189c070db54489bc399c8063d50497, State: Running, Role: LEADER
I20260619 06:49:45.498565 5448 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2189c070db54489bc399c8063d50497" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } }
I20260619 06:49:45.499521 5438 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:45.501420 5449 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2189c070db54489bc399c8063d50497. Latest consensus state: current_term: 1 leader_uuid: "c2189c070db54489bc399c8063d50497" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2189c070db54489bc399c8063d50497" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } } }
I20260619 06:49:45.501823 5449 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:45.502583 5450 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2189c070db54489bc399c8063d50497" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2189c070db54489bc399c8063d50497" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39323 } } }
I20260619 06:49:45.502894 5450 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:45.504525 5456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:45.509101 5456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:45.518483 5456 catalog_manager.cc:1347] Generated new cluster ID: c340dafc3112418b828e3ec3dedf565f
I20260619 06:49:45.518616 5456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:45.545094 5456 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:45.547067 5456 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:45.554559 5456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2189c070db54489bc399c8063d50497: Generated new TSK 0
I20260619 06:49:45.555837 5456 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:45.574646 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 5371
W20260619 06:49:45.575934 5433 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.47:60355
W20260619 06:49:45.576782 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 5371: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:49:45.576941 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 5371
W20260619 06:49:45.594475 5356 connection.cc:535] client connection to 127.0.0.1:39323 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39323): Connection reset by peer (error 104)
2026-06-19T06:49:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1924 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:45 dist-test-slave-5w2x krb5kdc[5469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:45 dist-test-slave-5w2x krb5kdc[5469](info): set up 2 sockets
Jun 19 06:49:45 dist-test-slave-5w2x krb5kdc[5469](info): commencing operation
krb5kdc: starting...
W20260619 06:49:46.685134 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:46 dist-test-slave-5w2x krb5kdc[5469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:49:46.941506 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40505
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:47.274554 5482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:47.274911 5482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:47.275067 5482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:47.282558 5482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:47.282734 5482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:47.282835 5482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:47.282941 5482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:47.283015 5482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:47.293251 5482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:47.294633 5482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:47.295940 5482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:47.308269 5490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:47.308291 5488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:47.308295 5487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:47.309082 5482 server_base.cc:1034] running on GCE node
I20260619 06:49:47.309948 5482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:47.311445 5482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:47.312670 5482 hybrid_clock.cc:648] HybridClock initialized: now 1781851787312621 us; error 44 us; skew 500 ppm
I20260619 06:49:47.313073 5482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:47 dist-test-slave-5w2x krb5kdc[5469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851787, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:47.320868 5482 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:49:47.322546 5482 webserver.cc:460] Webserver started at http://127.0.0.1:35629/ using document root <none> and password file <none>
I20260619 06:49:47.323293 5482 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:47.323450 5482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:47.323776 5482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:47.327069 5482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "5b14ee2083504efa96671a124064f4ae"
format_stamp: "Formatted at 2026-06-19 06:49:47 on dist-test-slave-5w2x"
server_key: "1725ea7337ea3d0ed5b61a76506f3224"
server_key_iv: "39e1912f029d6c8fda0586971b92b94a"
server_key_version: "encryptionkey@0"
I20260619 06:49:47.327912 5482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "5b14ee2083504efa96671a124064f4ae"
format_stamp: "Formatted at 2026-06-19 06:49:47 on dist-test-slave-5w2x"
server_key: "1725ea7337ea3d0ed5b61a76506f3224"
server_key_iv: "39e1912f029d6c8fda0586971b92b94a"
server_key_version: "encryptionkey@0"
I20260619 06:49:47.333463 5482 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260619 06:49:47.337144 5497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:47.338550 5482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 06:49:47.338740 5482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "5b14ee2083504efa96671a124064f4ae"
format_stamp: "Formatted at 2026-06-19 06:49:47 on dist-test-slave-5w2x"
server_key: "1725ea7337ea3d0ed5b61a76506f3224"
server_key_iv: "39e1912f029d6c8fda0586971b92b94a"
server_key_version: "encryptionkey@0"
I20260619 06:49:47.338954 5482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:47.344738 5482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:47.349009 5482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:47.349318 5482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:47.367224 5482 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40505
I20260619 06:49:47.367281 5548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40505 every 8 connection(s)
I20260619 06:49:47.368680 5482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:47.372413 5549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:47.376557 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 5482
I20260619 06:49:47.376739 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:47.377311 16701 external_mini_cluster.cc:1428] Setting key 3d0fc0591dc01724ff9c305c7a45180e
I20260619 06:49:47.383993 5549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae: Bootstrap starting.
I20260619 06:49:47.386595 5552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:47.379101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:48265 (local address 127.0.0.1:40505)
0619 06:49:47.379877 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:47.379893 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:47.379940 (+ 47us) server_negotiation.cc:407] Connection header received
0619 06:49:47.382274 (+ 2334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:47.382306 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:47.382772 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:47.383178 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:47.384601 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:47.384613 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:47.384631 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:47.384682 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:47.384766 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:47.384800 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:47.385047 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:47.385433 (+ 386us) server_negotiation.cc:299] Negotiation successful
0619 06:49:47.385647 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":348,"threads_started":1}
I20260619 06:49:47.387576 5549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:47.388697 5549 log.cc:824] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:47.390202 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:49:47.391486 5549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae: No bootstrap required, opened a new log
I20260619 06:49:47.397008 5552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:47.394442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:39685 (local address 127.0.0.1:40505)
0619 06:49:47.394619 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:47.394628 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:47.394969 (+ 341us) server_negotiation.cc:407] Connection header received
0619 06:49:47.395161 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:47.395169 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:47.395213 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:47.395311 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:47.396342 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:47.396348 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:47.396353 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:47.396371 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:47.396415 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:47.396430 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:47.396556 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:47.396669 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 06:49:47.396744 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 06:49:47.399937 5549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b14ee2083504efa96671a124064f4ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40505 } }
I20260619 06:49:47.400358 5549 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:47.400461 5549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b14ee2083504efa96671a124064f4ae, State: Initialized, Role: FOLLOWER
I20260619 06:49:47.401014 5549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b14ee2083504efa96671a124064f4ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40505 } }
I20260619 06:49:47.401281 5549 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:47.401415 5549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:47.401595 5549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:47.404248 5549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b14ee2083504efa96671a124064f4ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40505 } }
I20260619 06:49:47.404783 5549 leader_election.cc:302] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b14ee2083504efa96671a124064f4ae; no voters:
I20260619 06:49:47.405354 5549 leader_election.cc:288] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:47.405485 5559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:47.406131 5559 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [term 1 LEADER]: Becoming Leader. State: Replica: 5b14ee2083504efa96671a124064f4ae, State: Running, Role: LEADER
I20260619 06:49:47.406661 5559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b14ee2083504efa96671a124064f4ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40505 } }
I20260619 06:49:47.407466 5549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:47.409507 5561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b14ee2083504efa96671a124064f4ae. Latest consensus state: current_term: 1 leader_uuid: "5b14ee2083504efa96671a124064f4ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b14ee2083504efa96671a124064f4ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40505 } } }
I20260619 06:49:47.409830 5561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:47.409368 5560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b14ee2083504efa96671a124064f4ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b14ee2083504efa96671a124064f4ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40505 } } }
I20260619 06:49:47.410292 5560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:47.412642 5564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:47.418859 5564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:47.427592 5564 catalog_manager.cc:1347] Generated new cluster ID: bb1298805f524b979268d0a74d25c470
I20260619 06:49:47.427714 5564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:47.442181 5564 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:47.442734 5564 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:47.451550 5564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b14ee2083504efa96671a124064f4ae: Generated new TSK 0
I20260619 06:49:47.452533 5564 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:47.465355 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 5482
W20260619 06:49:47.466609 5544 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.47:48265
W20260619 06:49:47.467414 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 5482: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:49:47.467572 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 5482
2026-06-19T06:49:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1886 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:47 dist-test-slave-5w2x krb5kdc[5580](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:47 dist-test-slave-5w2x krb5kdc[5580](info): set up 2 sockets
Jun 19 06:49:47 dist-test-slave-5w2x krb5kdc[5580](info): commencing operation
krb5kdc: starting...
W20260619 06:49:48.576762 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:48 dist-test-slave-5w2x krb5kdc[5580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:49:48.827430 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41959
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:49.160183 5593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:49.160506 5593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:49.160643 5593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:49.168035 5593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:49.168188 5593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:49.168277 5593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:49.168391 5593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:49.168452 5593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:49.178728 5593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41959
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:49.180177 5593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:49.181542 5593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:49.193698 5598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:49.193607 5599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:49.193663 5601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:49.194913 5593 server_base.cc:1034] running on GCE node
I20260619 06:49:49.195897 5593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:49.197764 5593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:49.199018 5593 hybrid_clock.cc:648] HybridClock initialized: now 1781851789198951 us; error 57 us; skew 500 ppm
I20260619 06:49:49.199404 5593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:49 dist-test-slave-5w2x krb5kdc[5580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851789, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:49.202910 5593 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:49:49.204456 5593 webserver.cc:460] Webserver started at http://127.0.0.1:40653/ using document root <none> and password file <none>
I20260619 06:49:49.205156 5593 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:49.205289 5593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:49.205617 5593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:49.208683 5593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "7870fbabbbe0443189224f983326c449"
format_stamp: "Formatted at 2026-06-19 06:49:49 on dist-test-slave-5w2x"
server_key: "3c058caa8a09c5c170465fcf9e86f76d"
server_key_iv: "151268b477cb8955e8ddaa8d6452a8da"
server_key_version: "encryptionkey@0"
I20260619 06:49:49.209470 5593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "7870fbabbbe0443189224f983326c449"
format_stamp: "Formatted at 2026-06-19 06:49:49 on dist-test-slave-5w2x"
server_key: "3c058caa8a09c5c170465fcf9e86f76d"
server_key_iv: "151268b477cb8955e8ddaa8d6452a8da"
server_key_version: "encryptionkey@0"
I20260619 06:49:49.214848 5593 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:49:49.218423 5608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:49.219774 5593 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 06:49:49.219975 5593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "7870fbabbbe0443189224f983326c449"
format_stamp: "Formatted at 2026-06-19 06:49:49 on dist-test-slave-5w2x"
server_key: "3c058caa8a09c5c170465fcf9e86f76d"
server_key_iv: "151268b477cb8955e8ddaa8d6452a8da"
server_key_version: "encryptionkey@0"
I20260619 06:49:49.220207 5593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:49.234856 5593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:49.238775 5593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:49.239068 5593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:49.258178 5593 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41959
I20260619 06:49:49.258212 5659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41959 every 8 connection(s)
I20260619 06:49:49.260083 5593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:49.262758 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 5593
I20260619 06:49:49.262917 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:49.263477 16701 external_mini_cluster.cc:1428] Setting key 162fa680a023efeb5a6c75e5b4acdd47
I20260619 06:49:49.265030 5660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:49.278559 5660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449: Bootstrap starting.
Jun 19 06:49:49 dist-test-slave-5w2x krb5kdc[5580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 06:49:49.281999 5660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:49.283247 5660 log.cc:824] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:49.286173 5660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449: No bootstrap required, opened a new log
I20260619 06:49:49.288946 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:49.265546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:48391 (local address 127.0.0.1:41959)
0619 06:49:49.268691 (+ 3145us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:49.268705 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:49.268748 (+ 43us) server_negotiation.cc:407] Connection header received
0619 06:49:49.270497 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:49.270519 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:49.270963 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:49.271386 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:49.273149 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:49.274440 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:49.275741 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:49.276547 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:49.276750 (+ 203us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:49.281228 (+ 4478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:49.281236 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:49.281250 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:49.281287 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:49.284630 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:49.285489 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:49.285496 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:49.285502 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:49.285665 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:49.286132 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:49.286136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:49.286138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:49.286704 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:49.287042 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:49.287527 (+ 485us) server_negotiation.cc:299] Negotiation successful
0619 06:49:49.287866 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2923,"thread_start_us":2704,"threads_started":1}
I20260619 06:49:49.292999 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:49:49.294510 5660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7870fbabbbe0443189224f983326c449" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41959 } }
I20260619 06:49:49.294903 5660 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:49.295013 5660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7870fbabbbe0443189224f983326c449, State: Initialized, Role: FOLLOWER
I20260619 06:49:49.295707 5660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7870fbabbbe0443189224f983326c449" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41959 } }
I20260619 06:49:49.295962 5660 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:49.296116 5660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:49.296332 5660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:49.299553 5660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7870fbabbbe0443189224f983326c449" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41959 } }
I20260619 06:49:49.300288 5660 leader_election.cc:302] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7870fbabbbe0443189224f983326c449; no voters:
I20260619 06:49:49.301023 5660 leader_election.cc:288] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:49.301296 5670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:49.302371 5670 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [term 1 LEADER]: Becoming Leader. State: Replica: 7870fbabbbe0443189224f983326c449, State: Running, Role: LEADER
I20260619 06:49:49.303006 5670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7870fbabbbe0443189224f983326c449" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41959 } }
I20260619 06:49:49.303498 5660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:49.306783 5672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7870fbabbbe0443189224f983326c449. Latest consensus state: current_term: 1 leader_uuid: "7870fbabbbe0443189224f983326c449" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7870fbabbbe0443189224f983326c449" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41959 } } }
I20260619 06:49:49.307227 5672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:49.306782 5671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7870fbabbbe0443189224f983326c449" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7870fbabbbe0443189224f983326c449" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41959 } } }
I20260619 06:49:49.307924 5671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:49.312363 5678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:49.318256 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:49.297549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:59839 (local address 127.0.0.1:41959)
0619 06:49:49.297792 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:49.297798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:49.298172 (+ 374us) server_negotiation.cc:407] Connection header received
0619 06:49:49.298353 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:49.298359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:49.298425 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:49.298510 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:49.300576 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:49.301933 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:49.303871 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:49.304641 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:49.304813 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:49.309168 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:49.309179 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:49.309187 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:49.309244 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:49.313523 (+ 4279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:49.314693 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:49.314699 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:49.314702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:49.314853 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:49.315453 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:49.315458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:49.315461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:49.315887 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:49.316192 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:49.317638 (+ 1446us) server_negotiation.cc:299] Negotiation successful
0619 06:49:49.317828 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 06:49:49.318356 5678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:49.327981 5678 catalog_manager.cc:1347] Generated new cluster ID: 445ea1adde5f4b378edae487e0dcc61d
I20260619 06:49:49.328183 5678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:49.346161 5678 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:49.347572 5678 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:49.357476 5678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7870fbabbbe0443189224f983326c449: Generated new TSK 0
I20260619 06:49:49.358526 5678 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:49.373274 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 5593
I20260619 06:49:49.374475 5655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:49.467605 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 5593
W20260619 06:49:49.484592 5578 connection.cc:535] client connection to 127.0.0.1:41959 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41959): Connection reset by peer (error 104)
2026-06-19T06:49:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2002 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:49 dist-test-slave-5w2x krb5kdc[5692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:49 dist-test-slave-5w2x krb5kdc[5692](info): set up 2 sockets
Jun 19 06:49:49 dist-test-slave-5w2x krb5kdc[5692](info): commencing operation
krb5kdc: starting...
W20260619 06:49:50.581171 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.004s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:50 dist-test-slave-5w2x krb5kdc[5692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:49:50.832973 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32811
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:51.167811 5705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:51.168186 5705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:51.168352 5705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:51.175588 5705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:51.175747 5705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:51.175850 5705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:51.175963 5705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:51.176043 5705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:51.186161 5705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32811
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:51.187582 5705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:51.189011 5705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:51.200557 5710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:51.200605 5711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:51.200789 5713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:51.201251 5705 server_base.cc:1034] running on GCE node
I20260619 06:49:51.202031 5705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:51.203393 5705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:51.204603 5705 hybrid_clock.cc:648] HybridClock initialized: now 1781851791204563 us; error 37 us; skew 500 ppm
I20260619 06:49:51.204927 5705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:51 dist-test-slave-5w2x krb5kdc[5692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851791, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:51.211956 5705 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:49:51.213577 5705 webserver.cc:460] Webserver started at http://127.0.0.1:39833/ using document root <none> and password file <none>
I20260619 06:49:51.214270 5705 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:51.214398 5705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:51.214715 5705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:51.217875 5705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "3756d3983a6a4b3cab82cf13d9baa23e"
format_stamp: "Formatted at 2026-06-19 06:49:51 on dist-test-slave-5w2x"
server_key: "abe53598a2b2bbf53e64999465e92895"
server_key_iv: "8a302e8c4f8a481b42b3c20fe546f7b6"
server_key_version: "encryptionkey@0"
I20260619 06:49:51.218664 5705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "3756d3983a6a4b3cab82cf13d9baa23e"
format_stamp: "Formatted at 2026-06-19 06:49:51 on dist-test-slave-5w2x"
server_key: "abe53598a2b2bbf53e64999465e92895"
server_key_iv: "8a302e8c4f8a481b42b3c20fe546f7b6"
server_key_version: "encryptionkey@0"
I20260619 06:49:51.223927 5705 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260619 06:49:51.227451 5720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:51.228948 5705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:49:51.229131 5705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "3756d3983a6a4b3cab82cf13d9baa23e"
format_stamp: "Formatted at 2026-06-19 06:49:51 on dist-test-slave-5w2x"
server_key: "abe53598a2b2bbf53e64999465e92895"
server_key_iv: "8a302e8c4f8a481b42b3c20fe546f7b6"
server_key_version: "encryptionkey@0"
I20260619 06:49:51.229321 5705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:51.242955 5705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:51.247151 5705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:51.247444 5705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:51.269629 5705 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32811
I20260619 06:49:51.269668 5771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32811 every 8 connection(s)
I20260619 06:49:51.270992 5705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:51.274613 5772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:51.277575 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 5705
I20260619 06:49:51.277769 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:51.278298 16701 external_mini_cluster.cc:1428] Setting key 81cf1fb2889891df144eb3be4fc302bf
I20260619 06:49:51.287035 5772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e: Bootstrap starting.
I20260619 06:49:51.290843 5772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:51.291927 5772 log.cc:824] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:51.292044 5775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:51.280433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:33157 (local address 127.0.0.1:32811)
0619 06:49:51.281204 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:51.281216 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:51.281263 (+ 47us) server_negotiation.cc:407] Connection header received
0619 06:49:51.282973 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:51.282994 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:51.283427 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:51.283778 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:51.284378 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:51.285692 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:51.287556 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:51.288298 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:51.288425 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:51.289793 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:51.289801 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:51.289815 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:51.290332 (+ 517us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:51.290402 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:51.290430 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:51.290552 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:51.290839 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 06:49:51.291041 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":336,"threads_started":1}
I20260619 06:49:51.294902 5772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e: No bootstrap required, opened a new log
I20260619 06:49:51.296066 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:49:51.303249 5772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3756d3983a6a4b3cab82cf13d9baa23e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32811 } }
I20260619 06:49:51.303663 5772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:51.303767 5772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3756d3983a6a4b3cab82cf13d9baa23e, State: Initialized, Role: FOLLOWER
I20260619 06:49:51.304471 5772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3756d3983a6a4b3cab82cf13d9baa23e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32811 } }
I20260619 06:49:51.304749 5772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:51.304919 5772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:51.305114 5772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:51.306792 5775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:51.300481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:56909 (local address 127.0.0.1:32811)
0619 06:49:51.300688 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:51.300696 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:51.301036 (+ 340us) server_negotiation.cc:407] Connection header received
0619 06:49:51.301257 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:51.301262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:51.301331 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:51.301444 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:51.301953 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:51.302737 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:51.304194 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:51.304640 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:51.304789 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:51.306014 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:51.306018 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:51.306021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:51.306051 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:51.306111 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:51.306130 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:51.306266 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:51.306461 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 06:49:51.306518 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 06:49:51.307909 5772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3756d3983a6a4b3cab82cf13d9baa23e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32811 } }
I20260619 06:49:51.308490 5772 leader_election.cc:302] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3756d3983a6a4b3cab82cf13d9baa23e; no voters:
I20260619 06:49:51.309075 5772 leader_election.cc:288] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:51.309202 5782 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:51.311174 5772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:51.311180 5782 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [term 1 LEADER]: Becoming Leader. State: Replica: 3756d3983a6a4b3cab82cf13d9baa23e, State: Running, Role: LEADER
I20260619 06:49:51.311743 5782 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3756d3983a6a4b3cab82cf13d9baa23e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32811 } }
I20260619 06:49:51.314857 5783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3756d3983a6a4b3cab82cf13d9baa23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3756d3983a6a4b3cab82cf13d9baa23e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32811 } } }
I20260619 06:49:51.315055 5784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3756d3983a6a4b3cab82cf13d9baa23e. Latest consensus state: current_term: 1 leader_uuid: "3756d3983a6a4b3cab82cf13d9baa23e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3756d3983a6a4b3cab82cf13d9baa23e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32811 } } }
I20260619 06:49:51.315333 5783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:51.315374 5784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:51.317341 5791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:51.322309 5791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:51.330705 5791 catalog_manager.cc:1347] Generated new cluster ID: 732249a150604950a7568e2787cd717d
I20260619 06:49:51.330839 5791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:51.339360 5791 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:51.341279 5791 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:51.356668 5791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3756d3983a6a4b3cab82cf13d9baa23e: Generated new TSK 0
I20260619 06:49:51.357806 5791 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:51.377099 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 5705
W20260619 06:49:51.378419 5767 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.47:33157
W20260619 06:49:51.379479 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 5705: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:49:51.379655 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 5705
W20260619 06:49:51.402737 5690 connection.cc:535] client connection to 127.0.0.1:32811 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32811): Connection reset by peer (error 104)
2026-06-19T06:49:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1922 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:51 dist-test-slave-5w2x krb5kdc[5803](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:51 dist-test-slave-5w2x krb5kdc[5803](info): set up 2 sockets
Jun 19 06:49:51 dist-test-slave-5w2x krb5kdc[5803](info): commencing operation
krb5kdc: starting...
W20260619 06:49:52.508636 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:52 dist-test-slave-5w2x krb5kdc[5803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 06:49:52.755508 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42135
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:53.084789 5816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:53.085096 5816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:53.085280 5816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:53.092536 5816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:53.092639 5816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:53.092773 5816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:53.092877 5816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:53.092939 5816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:53.102713 5816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42135
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:53.103973 5816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:53.105319 5816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:53.117139 5824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:53.117355 5821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:53.117386 5822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:53.118589 5816 server_base.cc:1034] running on GCE node
I20260619 06:49:53.119403 5816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:53.120947 5816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:53.122184 5816 hybrid_clock.cc:648] HybridClock initialized: now 1781851793122136 us; error 65 us; skew 500 ppm
I20260619 06:49:53.122538 5816 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:53 dist-test-slave-5w2x krb5kdc[5803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851793, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:53.128538 5816 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 06:49:53.130007 5816 webserver.cc:460] Webserver started at http://127.0.0.1:34033/ using document root <none> and password file <none>
I20260619 06:49:53.130712 5816 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:53.130859 5816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:53.131167 5816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:53.134131 5816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "2eeeeface1114b8ea74179a27d1dc5fe"
format_stamp: "Formatted at 2026-06-19 06:49:53 on dist-test-slave-5w2x"
server_key: "73af0c9d34bef3a8f933eed05ea5de8a"
server_key_iv: "1bf9e7c08898a57aacf21afd2111c18e"
server_key_version: "encryptionkey@0"
I20260619 06:49:53.134893 5816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "2eeeeface1114b8ea74179a27d1dc5fe"
format_stamp: "Formatted at 2026-06-19 06:49:53 on dist-test-slave-5w2x"
server_key: "73af0c9d34bef3a8f933eed05ea5de8a"
server_key_iv: "1bf9e7c08898a57aacf21afd2111c18e"
server_key_version: "encryptionkey@0"
I20260619 06:49:53.139979 5816 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:49:53.143347 5831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:53.144649 5816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:49:53.144837 5816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "2eeeeface1114b8ea74179a27d1dc5fe"
format_stamp: "Formatted at 2026-06-19 06:49:53 on dist-test-slave-5w2x"
server_key: "73af0c9d34bef3a8f933eed05ea5de8a"
server_key_iv: "1bf9e7c08898a57aacf21afd2111c18e"
server_key_version: "encryptionkey@0"
I20260619 06:49:53.145057 5816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:53.150689 5816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:53.154932 5816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:53.155227 5816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:53.173251 5816 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42135
I20260619 06:49:53.173296 5882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42135 every 8 connection(s)
I20260619 06:49:53.174679 5816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:53.178413 5883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:53.179457 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 5816
I20260619 06:49:53.179633 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:53.180203 16701 external_mini_cluster.cc:1428] Setting key 598526b71e94d982d319c4fa748ff4a0
I20260619 06:49:53.188226 5886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:53.182054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:59609 (local address 127.0.0.1:42135)
0619 06:49:53.182852 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:53.182867 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:53.182907 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:49:53.184749 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:53.184775 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:53.185160 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:53.185528 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:53.186610 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:53.186616 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:53.186629 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:53.186644 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:53.186704 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:53.186728 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:53.186905 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:53.187005 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 06:49:53.187208 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":367,"threads_started":1}
I20260619 06:49:53.191636 5883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe: Bootstrap starting.
I20260619 06:49:53.191846 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:49:53.194967 5883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:53.196027 5883 log.cc:824] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:53.198659 5886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:53.196040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.47:55087 (local address 127.0.0.1:42135)
0619 06:49:53.196236 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:53.196243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:53.196770 (+ 527us) server_negotiation.cc:407] Connection header received
0619 06:49:53.196963 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:53.196968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:53.197031 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:53.197131 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:53.198045 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:53.198051 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:53.198055 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:53.198072 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:53.198117 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:53.198132 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:53.198243 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:53.198338 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 06:49:53.198397 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:49:53.198729 5883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe: No bootstrap required, opened a new log
I20260619 06:49:53.206142 5883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eeeeface1114b8ea74179a27d1dc5fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } }
I20260619 06:49:53.206501 5883 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:53.206614 5883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eeeeface1114b8ea74179a27d1dc5fe, State: Initialized, Role: FOLLOWER
I20260619 06:49:53.207204 5883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eeeeface1114b8ea74179a27d1dc5fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } }
I20260619 06:49:53.207438 5883 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:53.207590 5883 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:53.207762 5883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:53.209995 5883 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eeeeface1114b8ea74179a27d1dc5fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } }
I20260619 06:49:53.210582 5883 leader_election.cc:302] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2eeeeface1114b8ea74179a27d1dc5fe; no voters:
I20260619 06:49:53.211277 5883 leader_election.cc:288] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:53.211365 5893 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:53.212204 5893 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [term 1 LEADER]: Becoming Leader. State: Replica: 2eeeeface1114b8ea74179a27d1dc5fe, State: Running, Role: LEADER
I20260619 06:49:53.212819 5893 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eeeeface1114b8ea74179a27d1dc5fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } }
I20260619 06:49:53.213613 5883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:53.216076 5894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2eeeeface1114b8ea74179a27d1dc5fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eeeeface1114b8ea74179a27d1dc5fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } } }
I20260619 06:49:53.216928 5894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:53.216073 5895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2eeeeface1114b8ea74179a27d1dc5fe. Latest consensus state: current_term: 1 leader_uuid: "2eeeeface1114b8ea74179a27d1dc5fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eeeeface1114b8ea74179a27d1dc5fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42135 } } }
I20260619 06:49:53.217767 5895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:53.218505 5900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:53.224859 5900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:53.233952 5900 catalog_manager.cc:1347] Generated new cluster ID: 16abe3f0c0bd4bdc99c0f9a0e6b2edd4
I20260619 06:49:53.234066 5900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:53.289197 5900 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:53.289685 5900 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:53.313903 5900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2eeeeface1114b8ea74179a27d1dc5fe: Generated new TSK 0
I20260619 06:49:53.315086 5900 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:53.327980 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 5816
W20260619 06:49:53.329864 5878 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.47:59609
W20260619 06:49:53.330655 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 5816: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:49:53.330816 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 5816
2026-06-19T06:49:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1937 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:53 dist-test-slave-5w2x krb5kdc[5914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:53 dist-test-slave-5w2x krb5kdc[5914](info): set up 2 sockets
Jun 19 06:49:53 dist-test-slave-5w2x krb5kdc[5914](info): commencing operation
krb5kdc: starting...
W20260619 06:49:54.444638 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.009s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:54 dist-test-slave-5w2x krb5kdc[5914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:49:54.701634 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:43801
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:43963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:43801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:55.036289 5927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:55.036564 5927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:55.036751 5927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:55.043874 5927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:55.043990 5927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:55.044175 5927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:55.044275 5927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:55.044359 5927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:55.054104 5927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:43963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:43801
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:43801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:55.055418 5927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:55.056759 5927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:55.068208 5935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:55.068171 5932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:55.068248 5933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:55.069203 5927 server_base.cc:1034] running on GCE node
I20260619 06:49:55.069936 5927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:55.071233 5927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:55.072422 5927 hybrid_clock.cc:648] HybridClock initialized: now 1781851795072384 us; error 34 us; skew 500 ppm
I20260619 06:49:55.072880 5927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:55 dist-test-slave-5w2x krb5kdc[5914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851795, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:55.077158 5927 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:49:55.079063 5927 webserver.cc:460] Webserver started at http://127.16.79.126:35255/ using document root <none> and password file <none>
I20260619 06:49:55.079978 5927 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:55.080118 5927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:55.080533 5927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:55.084128 5927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "fa99a49331db435a8e061b3d15d0efa6"
format_stamp: "Formatted at 2026-06-19 06:49:55 on dist-test-slave-5w2x"
server_key: "306522b51ec70b3386118eddf8068857"
server_key_iv: "e24ff91a3f6fc662a992d867c02aad49"
server_key_version: "encryptionkey@0"
I20260619 06:49:55.085023 5927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "fa99a49331db435a8e061b3d15d0efa6"
format_stamp: "Formatted at 2026-06-19 06:49:55 on dist-test-slave-5w2x"
server_key: "306522b51ec70b3386118eddf8068857"
server_key_iv: "e24ff91a3f6fc662a992d867c02aad49"
server_key_version: "encryptionkey@0"
I20260619 06:49:55.090364 5927 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:49:55.093811 5942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:55.095088 5927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:49:55.095289 5927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "fa99a49331db435a8e061b3d15d0efa6"
format_stamp: "Formatted at 2026-06-19 06:49:55 on dist-test-slave-5w2x"
server_key: "306522b51ec70b3386118eddf8068857"
server_key_iv: "e24ff91a3f6fc662a992d867c02aad49"
server_key_version: "encryptionkey@0"
I20260619 06:49:55.095487 5927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:55.111146 5927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:55.115031 5927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:55.115312 5927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:55.133720 5927 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:43801
I20260619 06:49:55.133728 5993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:43801 every 8 connection(s)
I20260619 06:49:55.135129 5927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:55.136811 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 5927
I20260619 06:49:55.136988 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:55.137481 16701 external_mini_cluster.cc:1428] Setting key 1a4f089f34ed2119ac3ba4f7d22ca27d
I20260619 06:49:55.139533 5994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Jun 19 06:49:55 dist-test-slave-5w2x krb5kdc[5914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:49:55.153129 5994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6: Bootstrap starting.
I20260619 06:49:55.157285 5994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:55.158704 5994 log.cc:824] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:55.162268 5994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6: No bootstrap required, opened a new log
I20260619 06:49:55.163327 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:55.139715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50047 (local address 127.16.79.126:43801)
0619 06:49:55.140555 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:55.140572 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:55.140623 (+ 51us) server_negotiation.cc:407] Connection header received
0619 06:49:55.142791 (+ 2168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:55.142818 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:55.143363 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:55.143808 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:55.145348 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:55.147403 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:55.148732 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:55.149818 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:55.149996 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:55.154188 (+ 4192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:55.154199 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:55.154218 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:55.154268 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:55.158357 (+ 4089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:55.159322 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:55.159332 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:55.159341 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:55.159538 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:55.160118 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:55.160122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:55.160126 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:55.160840 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:55.161187 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:55.161511 (+ 324us) server_negotiation.cc:299] Negotiation successful
0619 06:49:55.161854 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":375,"threads_started":1}
I20260619 06:49:55.167440 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:49:55.171161 5994 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa99a49331db435a8e061b3d15d0efa6" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43801 } }
I20260619 06:49:55.171581 5994 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:55.171741 5994 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa99a49331db435a8e061b3d15d0efa6, State: Initialized, Role: FOLLOWER
I20260619 06:49:55.172411 5994 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa99a49331db435a8e061b3d15d0efa6" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43801 } }
I20260619 06:49:55.172757 5994 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:55.172923 5994 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:55.173126 5994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:55.175956 5994 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa99a49331db435a8e061b3d15d0efa6" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43801 } }
I20260619 06:49:55.176512 5994 leader_election.cc:302] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa99a49331db435a8e061b3d15d0efa6; no voters:
I20260619 06:49:55.177232 5994 leader_election.cc:288] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:55.177379 6004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:55.179057 6004 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [term 1 LEADER]: Becoming Leader. State: Replica: fa99a49331db435a8e061b3d15d0efa6, State: Running, Role: LEADER
I20260619 06:49:55.179725 6004 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa99a49331db435a8e061b3d15d0efa6" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43801 } }
I20260619 06:49:55.180011 5994 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:55.187014 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:55.171698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59373 (local address 127.16.79.126:43801)
0619 06:49:55.171890 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:55.171896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:55.172476 (+ 580us) server_negotiation.cc:407] Connection header received
0619 06:49:55.172617 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:55.172622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:55.172704 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:55.172824 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:55.174343 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:55.175505 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:55.176856 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:55.177496 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:55.177655 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:55.180654 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:55.180660 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:55.180664 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:49:55.180696 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:55.183914 (+ 3218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:55.184771 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:55.184776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:55.184780 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:55.184907 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:49:55.185388 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:49:55.185394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:49:55.185397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:49:55.185774 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:55.186243 (+ 469us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:55.186440 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 06:49:55.186617 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 06:49:55.189018 6005 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa99a49331db435a8e061b3d15d0efa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa99a49331db435a8e061b3d15d0efa6" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43801 } } }
I20260619 06:49:55.189252 6006 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa99a49331db435a8e061b3d15d0efa6. Latest consensus state: current_term: 1 leader_uuid: "fa99a49331db435a8e061b3d15d0efa6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa99a49331db435a8e061b3d15d0efa6" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 43801 } } }
I20260619 06:49:55.189463 6005 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:55.189491 6006 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:55.193269 6014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:55.198109 6014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:55.206532 6014 catalog_manager.cc:1347] Generated new cluster ID: 7b43475613e8421c9c313b7b26256c67
I20260619 06:49:55.206672 6014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:55.226766 6014 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:55.228070 6014 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:55.237516 6014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa99a49331db435a8e061b3d15d0efa6: Generated new TSK 0
I20260619 06:49:55.238341 6014 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:55.247398 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 5927
I20260619 06:49:55.248565 5989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:49:55.344250 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 5927
W20260619 06:49:55.361424 5912 connection.cc:535] client connection to 127.16.79.126:43801 recv error: Network error: failed to read from TLS socket (remote: 127.16.79.126:43801): Connection reset by peer (error 104)
2026-06-19T06:49:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2016 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:55 dist-test-slave-5w2x krb5kdc[6026](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:55 dist-test-slave-5w2x krb5kdc[6026](info): set up 2 sockets
Jun 19 06:49:55 dist-test-slave-5w2x krb5kdc[6026](info): commencing operation
krb5kdc: starting...
W20260619 06:49:56.456553 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:56 dist-test-slave-5w2x krb5kdc[6026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:49:56.705554 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:35153
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:37973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:35153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:57.030588 6039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:57.031014 6039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:57.031188 6039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:57.038833 6039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:57.038998 6039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:57.039100 6039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:57.039207 6039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:57.039283 6039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:57.049543 6039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:37973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:35153
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:35153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:57.050899 6039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:57.052335 6039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:57.064069 6044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:57.064126 6045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:57.064090 6047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:57.064967 6039 server_base.cc:1034] running on GCE node
I20260619 06:49:57.065825 6039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:57.067269 6039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:57.068499 6039 hybrid_clock.cc:648] HybridClock initialized: now 1781851797068459 us; error 52 us; skew 500 ppm
I20260619 06:49:57.068862 6039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:57 dist-test-slave-5w2x krb5kdc[6026](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851797, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:57.072042 6039 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:49:57.073648 6039 webserver.cc:460] Webserver started at http://127.16.79.126:40627/ using document root <none> and password file <none>
I20260619 06:49:57.074383 6039 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:57.074534 6039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:57.074847 6039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:57.077986 6039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "8597777f978f4e83a8a93711a1e22879"
format_stamp: "Formatted at 2026-06-19 06:49:57 on dist-test-slave-5w2x"
server_key: "6da9a1337ac2fbd9b374fa762bfafdac"
server_key_iv: "6395854ee037db2b768bb2f07fcf6ef2"
server_key_version: "encryptionkey@0"
I20260619 06:49:57.078815 6039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "8597777f978f4e83a8a93711a1e22879"
format_stamp: "Formatted at 2026-06-19 06:49:57 on dist-test-slave-5w2x"
server_key: "6da9a1337ac2fbd9b374fa762bfafdac"
server_key_iv: "6395854ee037db2b768bb2f07fcf6ef2"
server_key_version: "encryptionkey@0"
I20260619 06:49:57.083957 6039 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260619 06:49:57.087363 6054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:57.089103 6039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:49:57.089336 6039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "8597777f978f4e83a8a93711a1e22879"
format_stamp: "Formatted at 2026-06-19 06:49:57 on dist-test-slave-5w2x"
server_key: "6da9a1337ac2fbd9b374fa762bfafdac"
server_key_iv: "6395854ee037db2b768bb2f07fcf6ef2"
server_key_version: "encryptionkey@0"
I20260619 06:49:57.089576 6039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:57.102952 6039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:57.106999 6039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:57.107311 6039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:57.126629 6039 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:35153
I20260619 06:49:57.126636 6105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:35153 every 8 connection(s)
I20260619 06:49:57.128206 6039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:57.129323 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 6039
I20260619 06:49:57.129508 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:57.130059 16701 external_mini_cluster.cc:1428] Setting key 47838b1950e8d1f3995ed05c01d0d786
I20260619 06:49:57.132928 6106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:57.143430 6109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:57.131994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50555 (local address 127.16.79.126:35153)
0619 06:49:57.132782 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:57.132794 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:57.132838 (+ 44us) server_negotiation.cc:407] Connection header received
0619 06:49:57.134644 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:57.134665 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:57.135059 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:57.135495 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:57.136056 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:57.137519 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:57.138851 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:57.139541 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:57.139734 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:57.141075 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:57.141082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:57.141099 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:57.141694 (+ 595us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:57.141773 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:57.141801 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:57.141983 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:57.142312 (+ 329us) server_negotiation.cc:299] Negotiation successful
0619 06:49:57.142471 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":349,"threads_started":1}
I20260619 06:49:57.144060 6106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879: Bootstrap starting.
I20260619 06:49:57.147419 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:49:57.147550 6106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:57.148641 6106 log.cc:824] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:57.151473 6106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879: No bootstrap required, opened a new log
I20260619 06:49:57.158882 6109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:57.151885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47097 (local address 127.16.79.126:35153)
0619 06:49:57.152126 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:57.152161 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:57.152515 (+ 354us) server_negotiation.cc:407] Connection header received
0619 06:49:57.152724 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:57.152727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:57.152785 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:57.152861 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:57.153500 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:57.154311 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:57.156251 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:49:57.156734 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:49:57.156844 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:49:57.158084 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:57.158089 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:57.158092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:57.158110 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:57.158157 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:57.158173 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:57.158266 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:57.158461 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 06:49:57.158512 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 06:49:57.161224 6106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8597777f978f4e83a8a93711a1e22879" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 35153 } }
I20260619 06:49:57.161629 6106 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:57.161741 6106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8597777f978f4e83a8a93711a1e22879, State: Initialized, Role: FOLLOWER
I20260619 06:49:57.162310 6106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8597777f978f4e83a8a93711a1e22879" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 35153 } }
I20260619 06:49:57.162554 6106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:57.162700 6106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:57.162874 6106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:57.165158 6106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8597777f978f4e83a8a93711a1e22879" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 35153 } }
I20260619 06:49:57.165670 6106 leader_election.cc:302] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8597777f978f4e83a8a93711a1e22879; no voters:
I20260619 06:49:57.166273 6106 leader_election.cc:288] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:57.166425 6116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:57.167142 6116 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [term 1 LEADER]: Becoming Leader. State: Replica: 8597777f978f4e83a8a93711a1e22879, State: Running, Role: LEADER
I20260619 06:49:57.167660 6116 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8597777f978f4e83a8a93711a1e22879" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 35153 } }
I20260619 06:49:57.168463 6106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:57.170545 6118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8597777f978f4e83a8a93711a1e22879. Latest consensus state: current_term: 1 leader_uuid: "8597777f978f4e83a8a93711a1e22879" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8597777f978f4e83a8a93711a1e22879" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 35153 } } }
I20260619 06:49:57.170939 6118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:57.171749 6117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8597777f978f4e83a8a93711a1e22879" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8597777f978f4e83a8a93711a1e22879" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 35153 } } }
I20260619 06:49:57.172178 6117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879 [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:57.173038 6121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:57.178236 6121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:57.187500 6121 catalog_manager.cc:1347] Generated new cluster ID: a344a050a20e4e5eb4e1194a81f62274
I20260619 06:49:57.187629 6121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:57.211197 6121 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:57.213114 6121 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:57.230594 6121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8597777f978f4e83a8a93711a1e22879: Generated new TSK 0
I20260619 06:49:57.231717 6121 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:57.241756 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 6039
W20260619 06:49:57.243000 6101 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50555
W20260619 06:49:57.243752 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 6039: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:49:57.243899 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 6039
W20260619 06:49:57.263271 6024 connection.cc:535] client connection to 127.16.79.126:35153 recv error: Network error: failed to read from TLS socket (remote: 127.16.79.126:35153): Connection reset by peer (error 104)
2026-06-19T06:49:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1902 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:57 dist-test-slave-5w2x krb5kdc[6137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:57 dist-test-slave-5w2x krb5kdc[6137](info): set up 2 sockets
Jun 19 06:49:57 dist-test-slave-5w2x krb5kdc[6137](info): commencing operation
krb5kdc: starting...
W20260619 06:49:58.356702 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:49:58 dist-test-slave-5w2x krb5kdc[6137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851798, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:49:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:49:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:49:58.613590 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:39857
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:37323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:39857
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:49:58.938361 6150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:49:58.938699 6150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:49:58.938871 6150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:49:58.945991 6150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:49:58.946157 6150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:49:58.946260 6150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:49:58.946383 6150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:49:58.946460 6150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:49:58.956071 6150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:37323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:39857
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:39857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:49:58.957391 6150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:49:58.958624 6150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:49:58.970083 6155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:58.970043 6158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:49:58.970118 6156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:49:58.971280 6150 server_base.cc:1034] running on GCE node
I20260619 06:49:58.972020 6150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:49:58.973429 6150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:49:58.974627 6150 hybrid_clock.cc:648] HybridClock initialized: now 1781851798974579 us; error 36 us; skew 500 ppm
I20260619 06:49:58.974967 6150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:49:58 dist-test-slave-5w2x krb5kdc[6137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851798, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:49:58.981050 6150 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:49:58.982712 6150 webserver.cc:460] Webserver started at http://127.16.79.126:38267/ using document root <none> and password file <none>
I20260619 06:49:58.983394 6150 fs_manager.cc:359] Metadata directory not provided
I20260619 06:49:58.983523 6150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:49:58.983836 6150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:49:58.986932 6150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "53d93a36b4ad4178b10f7fbdcaceebea"
format_stamp: "Formatted at 2026-06-19 06:49:58 on dist-test-slave-5w2x"
server_key: "8badcaf752f87fb725c806129403f108"
server_key_iv: "46b68089df10897d229213923922fed6"
server_key_version: "encryptionkey@0"
I20260619 06:49:58.987684 6150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "53d93a36b4ad4178b10f7fbdcaceebea"
format_stamp: "Formatted at 2026-06-19 06:49:58 on dist-test-slave-5w2x"
server_key: "8badcaf752f87fb725c806129403f108"
server_key_iv: "46b68089df10897d229213923922fed6"
server_key_version: "encryptionkey@0"
I20260619 06:49:58.992769 6150 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 06:49:58.996109 6165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:49:58.997531 6150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260619 06:49:58.997722 6150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "53d93a36b4ad4178b10f7fbdcaceebea"
format_stamp: "Formatted at 2026-06-19 06:49:58 on dist-test-slave-5w2x"
server_key: "8badcaf752f87fb725c806129403f108"
server_key_iv: "46b68089df10897d229213923922fed6"
server_key_version: "encryptionkey@0"
I20260619 06:49:58.997921 6150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:49:59.003533 6150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:49:59.007731 6150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:49:59.008024 6150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:49:59.025928 6150 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:39857
I20260619 06:49:59.025986 6216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:39857 every 8 connection(s)
I20260619 06:49:59.027274 6150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:49:59.027925 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 6150
I20260619 06:49:59.028175 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:49:59.028749 16701 external_mini_cluster.cc:1428] Setting key a187e0dd78d2559d0fe22c38be29db22
I20260619 06:49:59.031759 6217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:49:59.037341 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:59.030513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54553 (local address 127.16.79.126:39857)
0619 06:49:59.031269 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:59.031286 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:59.031338 (+ 52us) server_negotiation.cc:407] Connection header received
0619 06:49:59.033608 (+ 2270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:59.033634 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:59.034046 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:59.034472 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:59.035458 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:59.035467 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:59.035488 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:59.035509 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:59.035586 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:59.035610 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:59.035829 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:59.035966 (+ 137us) server_negotiation.cc:299] Negotiation successful
0619 06:49:59.036171 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":292,"threads_started":1}
I20260619 06:49:59.041118 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:49:59.045944 6217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea: Bootstrap starting.
I20260619 06:49:59.048993 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:49:59.046522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43517 (local address 127.16.79.126:39857)
0619 06:49:59.046688 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 06:49:59.046694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:49:59.047042 (+ 348us) server_negotiation.cc:407] Connection header received
0619 06:49:59.047188 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:49:59.047192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:49:59.047248 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:49:59.047351 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:49:59.048388 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:49:59.048393 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:49:59.048398 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:49:59.048419 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:49:59.048463 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:49:59.048479 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:49:59.048589 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 06:49:59.048671 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 06:49:59.048723 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 06:49:59.049846 6217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea: Neither blocks nor log segments found. Creating new log.
I20260619 06:49:59.050942 6217 log.cc:824] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea: Log is configured to *not* fsync() on all Append() calls
I20260619 06:49:59.053666 6217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea: No bootstrap required, opened a new log
I20260619 06:49:59.060633 6217 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d93a36b4ad4178b10f7fbdcaceebea" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39857 } }
I20260619 06:49:59.061002 6217 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:49:59.061075 6217 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53d93a36b4ad4178b10f7fbdcaceebea, State: Initialized, Role: FOLLOWER
I20260619 06:49:59.061558 6217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d93a36b4ad4178b10f7fbdcaceebea" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39857 } }
I20260619 06:49:59.061739 6217 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:49:59.061836 6217 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:49:59.061949 6217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:49:59.063961 6217 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d93a36b4ad4178b10f7fbdcaceebea" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39857 } }
I20260619 06:49:59.064468 6217 leader_election.cc:302] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53d93a36b4ad4178b10f7fbdcaceebea; no voters:
I20260619 06:49:59.065007 6217 leader_election.cc:288] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:49:59.065172 6227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:49:59.065872 6227 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [term 1 LEADER]: Becoming Leader. State: Replica: 53d93a36b4ad4178b10f7fbdcaceebea, State: Running, Role: LEADER
I20260619 06:49:59.066488 6227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d93a36b4ad4178b10f7fbdcaceebea" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39857 } }
I20260619 06:49:59.067158 6217 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:49:59.069581 6228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53d93a36b4ad4178b10f7fbdcaceebea. Latest consensus state: current_term: 1 leader_uuid: "53d93a36b4ad4178b10f7fbdcaceebea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d93a36b4ad4178b10f7fbdcaceebea" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39857 } } }
I20260619 06:49:59.070102 6228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:59.069657 6229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53d93a36b4ad4178b10f7fbdcaceebea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53d93a36b4ad4178b10f7fbdcaceebea" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 39857 } } }
I20260619 06:49:59.070744 6229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea [sys.catalog]: This master's current role is: LEADER
I20260619 06:49:59.072292 6234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:49:59.077785 6234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:49:59.086402 6234 catalog_manager.cc:1347] Generated new cluster ID: 19d2172580de44b8ad98aaaf6b57057b
I20260619 06:49:59.086510 6234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:49:59.111994 6234 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:49:59.112448 6234 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:49:59.125914 6234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53d93a36b4ad4178b10f7fbdcaceebea: Generated new TSK 0
I20260619 06:49:59.126843 6234 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:49:59.137421 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 6150
W20260619 06:49:59.138542 6212 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54553
W20260619 06:49:59.139248 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 6150: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:49:59.139406 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 6150
2026-06-19T06:49:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1892 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:49:59 dist-test-slave-5w2x krb5kdc[6248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:49:59 dist-test-slave-5w2x krb5kdc[6248](info): set up 2 sockets
Jun 19 06:49:59 dist-test-slave-5w2x krb5kdc[6248](info): commencing operation
krb5kdc: starting...
W20260619 06:50:00.248867 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:50:00 dist-test-slave-5w2x krb5kdc[6248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:50:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:50:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:50:00.504702 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:33365
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:35995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:33365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:00.841975 6261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:00.842283 6261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:00.842478 6261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:00.850039 6261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:00.850190 6261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:00.850289 6261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:00.850394 6261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:00.850471 6261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:00.860504 6261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:35995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:33365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:33365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:00.861817 6261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:00.863154 6261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:00.874987 6266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:00.875087 6267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:00.875110 6269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:00.875988 6261 server_base.cc:1034] running on GCE node
I20260619 06:50:00.876812 6261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:00.878223 6261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:00.879443 6261 hybrid_clock.cc:648] HybridClock initialized: now 1781851800879397 us; error 41 us; skew 500 ppm
I20260619 06:50:00.879817 6261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:00 dist-test-slave-5w2x krb5kdc[6248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851800, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:00.882929 6261 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:50:00.884639 6261 webserver.cc:460] Webserver started at http://127.16.79.126:32879/ using document root <none> and password file <none>
I20260619 06:50:00.885413 6261 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:00.885569 6261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:00.885885 6261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:00.889153 6261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "8cd99feaf11147b59e2942a7ad422ca5"
format_stamp: "Formatted at 2026-06-19 06:50:00 on dist-test-slave-5w2x"
server_key: "458abac845f3a2dcb9a9888fd8bc0d39"
server_key_iv: "1f9e0a9ebc367ad4a07d911455f3b871"
server_key_version: "encryptionkey@0"
I20260619 06:50:00.889959 6261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "8cd99feaf11147b59e2942a7ad422ca5"
format_stamp: "Formatted at 2026-06-19 06:50:00 on dist-test-slave-5w2x"
server_key: "458abac845f3a2dcb9a9888fd8bc0d39"
server_key_iv: "1f9e0a9ebc367ad4a07d911455f3b871"
server_key_version: "encryptionkey@0"
I20260619 06:50:00.895462 6261 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260619 06:50:00.899120 6276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:00.900584 6261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:50:00.900784 6261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "8cd99feaf11147b59e2942a7ad422ca5"
format_stamp: "Formatted at 2026-06-19 06:50:00 on dist-test-slave-5w2x"
server_key: "458abac845f3a2dcb9a9888fd8bc0d39"
server_key_iv: "1f9e0a9ebc367ad4a07d911455f3b871"
server_key_version: "encryptionkey@0"
I20260619 06:50:00.900995 6261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:00.918526 6261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:00.922849 6261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:00.923185 6261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:00.942879 6261 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:33365
I20260619 06:50:00.942909 6327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:33365 every 8 connection(s)
I20260619 06:50:00.944410 6261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:50:00.948314 6328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:00.950181 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 6261
I20260619 06:50:00.950372 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:50:00.950814 16701 external_mini_cluster.cc:1428] Setting key 6fa090e26fd988f69383a2a5f2962713
I20260619 06:50:00.963423 6328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5: Bootstrap starting.
Jun 19 06:50:00 dist-test-slave-5w2x krb5kdc[6248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:50:00.966952 6328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:00.968482 6328 log.cc:824] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:00.972030 6328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5: No bootstrap required, opened a new log
I20260619 06:50:00.976070 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:00.952790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39469 (local address 127.16.79.126:33365)
0619 06:50:00.953694 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:00.953710 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:00.953767 (+ 57us) server_negotiation.cc:407] Connection header received
0619 06:50:00.955524 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:00.955563 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:00.956209 (+ 646us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:00.956688 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:00.958101 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:00.960270 (+ 2169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:00.961610 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:00.962686 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:00.962874 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:00.967213 (+ 4339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:00.967225 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:00.967248 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:00.967306 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:00.971404 (+ 4098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:00.972176 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:00.972184 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:00.972193 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:00.972411 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:00.972790 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:00.972795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:00.972798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:00.973523 (+ 725us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:00.973824 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:00.974161 (+ 337us) server_negotiation.cc:299] Negotiation successful
0619 06:50:00.974643 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":379,"threads_started":1}
I20260619 06:50:00.980234 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:50:00.981989 6328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd99feaf11147b59e2942a7ad422ca5" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 33365 } }
I20260619 06:50:00.982366 6328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:00.982494 6328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cd99feaf11147b59e2942a7ad422ca5, State: Initialized, Role: FOLLOWER
I20260619 06:50:00.983142 6328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd99feaf11147b59e2942a7ad422ca5" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 33365 } }
I20260619 06:50:00.983389 6328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:50:00.983534 6328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:50:00.983727 6328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:00.987198 6328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd99feaf11147b59e2942a7ad422ca5" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 33365 } }
I20260619 06:50:00.987846 6328 leader_election.cc:302] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cd99feaf11147b59e2942a7ad422ca5; no voters:
I20260619 06:50:00.988602 6328 leader_election.cc:288] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:50:00.988715 6338 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:00.990444 6338 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [term 1 LEADER]: Becoming Leader. State: Replica: 8cd99feaf11147b59e2942a7ad422ca5, State: Running, Role: LEADER
I20260619 06:50:00.991045 6338 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd99feaf11147b59e2942a7ad422ca5" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 33365 } }
I20260619 06:50:00.991250 6328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:01.000166 6331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:00.984747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51971 (local address 127.16.79.126:33365)
0619 06:50:00.985015 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:00.985024 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:00.985359 (+ 335us) server_negotiation.cc:407] Connection header received
0619 06:50:00.985548 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:00.985552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:00.985621 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:00.985710 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:00.987612 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:00.988484 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:00.990230 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:00.990943 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:00.991089 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:00.993913 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:00.993918 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:00.993921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:00.993954 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:00.997212 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:00.997956 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:00.997962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:00.997965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:00.998112 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:00.998496 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:00.998503 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:00.998506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:00.998857 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:00.999163 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:00.999481 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 06:50:00.999733 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 06:50:01.001194 6340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cd99feaf11147b59e2942a7ad422ca5. Latest consensus state: current_term: 1 leader_uuid: "8cd99feaf11147b59e2942a7ad422ca5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd99feaf11147b59e2942a7ad422ca5" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 33365 } } }
I20260619 06:50:01.001677 6340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:01.001194 6339 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cd99feaf11147b59e2942a7ad422ca5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cd99feaf11147b59e2942a7ad422ca5" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 33365 } } }
I20260619 06:50:01.002219 6339 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:01.006415 6348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:50:01.010717 6348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:50:01.018981 6348 catalog_manager.cc:1347] Generated new cluster ID: 951da9fba6754443b6b049f032954130
I20260619 06:50:01.019102 6348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:50:01.040256 6348 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:50:01.042150 6348 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:50:01.050467 6348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cd99feaf11147b59e2942a7ad422ca5: Generated new TSK 0
I20260619 06:50:01.051469 6348 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:50:01.069190 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 6261
I20260619 06:50:01.070526 6323 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:01.163250 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 6261
2026-06-19T06:50:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2020 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:50:01 dist-test-slave-5w2x krb5kdc[6360](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:50:01 dist-test-slave-5w2x krb5kdc[6360](info): set up 2 sockets
Jun 19 06:50:01 dist-test-slave-5w2x krb5kdc[6360](info): commencing operation
krb5kdc: starting...
W20260619 06:50:02.288708 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:50:02 dist-test-slave-5w2x krb5kdc[6360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:50:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:50:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:50:02.544721 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:35133
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:35133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:02.876732 6373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:02.877053 6373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:02.877233 6373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:02.884704 6373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:02.884833 6373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:02.884963 6373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:02.885066 6373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:02.885149 6373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:02.896059 6373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:35133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:35133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:02.897545 6373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:02.898931 6373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:02.911074 6378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:02.911047 6379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:02.911124 6381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:02.912240 6373 server_base.cc:1034] running on GCE node
I20260619 06:50:02.913003 6373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:02.914446 6373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:02.915704 6373 hybrid_clock.cc:648] HybridClock initialized: now 1781851802915658 us; error 51 us; skew 500 ppm
I20260619 06:50:02.916087 6373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:02 dist-test-slave-5w2x krb5kdc[6360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851802, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:02.919219 6373 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:50:02.920734 6373 webserver.cc:460] Webserver started at http://127.16.79.126:41041/ using document root <none> and password file <none>
I20260619 06:50:02.921429 6373 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:02.921566 6373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:02.921867 6373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:02.924924 6373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "c6dbb40fa7854e3e88dd6d5fce368d49"
format_stamp: "Formatted at 2026-06-19 06:50:02 on dist-test-slave-5w2x"
server_key: "46b2c1c1816ba46b919d74330b7acd95"
server_key_iv: "e8cad0052201e1c37659035d8d2571e6"
server_key_version: "encryptionkey@0"
I20260619 06:50:02.925736 6373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "c6dbb40fa7854e3e88dd6d5fce368d49"
format_stamp: "Formatted at 2026-06-19 06:50:02 on dist-test-slave-5w2x"
server_key: "46b2c1c1816ba46b919d74330b7acd95"
server_key_iv: "e8cad0052201e1c37659035d8d2571e6"
server_key_version: "encryptionkey@0"
I20260619 06:50:02.931028 6373 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260619 06:50:02.934689 6388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:02.936447 6373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260619 06:50:02.936643 6373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "c6dbb40fa7854e3e88dd6d5fce368d49"
format_stamp: "Formatted at 2026-06-19 06:50:02 on dist-test-slave-5w2x"
server_key: "46b2c1c1816ba46b919d74330b7acd95"
server_key_iv: "e8cad0052201e1c37659035d8d2571e6"
server_key_version: "encryptionkey@0"
I20260619 06:50:02.936862 6373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:02.960901 6373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:02.965088 6373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:02.965391 6373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:02.984107 6373 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:35133
I20260619 06:50:02.984126 6439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:35133 every 8 connection(s)
I20260619 06:50:02.985503 6373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:50:02.989285 6440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:02.989586 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 6373
I20260619 06:50:02.989756 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:50:02.990276 16701 external_mini_cluster.cc:1428] Setting key 6c98ebebab418e41bbb75e192150e7bf
I20260619 06:50:03.003219 6440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49: Bootstrap starting.
I20260619 06:50:03.003552 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:02.992092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56221 (local address 127.16.79.126:35133)
0619 06:50:02.992973 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:02.992985 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:02.993027 (+ 42us) server_negotiation.cc:407] Connection header received
0619 06:50:02.994499 (+ 1472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:02.994518 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:02.995001 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:02.995341 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:02.995966 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:02.997379 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:02.999182 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:02.999884 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:03.000016 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:03.001293 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:03.001301 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:03.001317 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:50:03.001838 (+ 521us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:03.001908 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:50:03.001931 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:03.002023 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:03.002250 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 06:50:03.002530 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":378,"threads_started":1}
I20260619 06:50:03.006469 6440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:03.007329 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:50:03.007709 6440 log.cc:824] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:03.010504 6440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49: No bootstrap required, opened a new log
I20260619 06:50:03.018554 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:03.011700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51473 (local address 127.16.79.126:35133)
0619 06:50:03.011889 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:03.011894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:03.012703 (+ 809us) server_negotiation.cc:407] Connection header received
0619 06:50:03.012878 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:03.012888 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:03.012970 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:03.013078 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:03.013547 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:03.014634 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:03.015886 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:03.016648 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:03.016781 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:03.017756 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:03.017762 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:03.017768 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:50:03.017796 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:03.017849 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:50:03.017868 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:03.017963 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:03.018088 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 06:50:03.018191 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 06:50:03.019711 6440 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dbb40fa7854e3e88dd6d5fce368d49" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 35133 } }
I20260619 06:50:03.020215 6440 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:03.020303 6440 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6dbb40fa7854e3e88dd6d5fce368d49, State: Initialized, Role: FOLLOWER
I20260619 06:50:03.020802 6440 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dbb40fa7854e3e88dd6d5fce368d49" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 35133 } }
I20260619 06:50:03.021004 6440 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:50:03.021097 6440 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:50:03.021288 6440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:03.023619 6440 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dbb40fa7854e3e88dd6d5fce368d49" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 35133 } }
I20260619 06:50:03.024076 6440 leader_election.cc:302] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6dbb40fa7854e3e88dd6d5fce368d49; no voters:
I20260619 06:50:03.024657 6440 leader_election.cc:288] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:50:03.024814 6450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:03.025467 6450 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [term 1 LEADER]: Becoming Leader. State: Replica: c6dbb40fa7854e3e88dd6d5fce368d49, State: Running, Role: LEADER
I20260619 06:50:03.026078 6450 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dbb40fa7854e3e88dd6d5fce368d49" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 35133 } }
I20260619 06:50:03.026850 6440 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:03.029264 6452 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6dbb40fa7854e3e88dd6d5fce368d49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dbb40fa7854e3e88dd6d5fce368d49" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 35133 } } }
I20260619 06:50:03.029314 6451 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6dbb40fa7854e3e88dd6d5fce368d49. Latest consensus state: current_term: 1 leader_uuid: "c6dbb40fa7854e3e88dd6d5fce368d49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6dbb40fa7854e3e88dd6d5fce368d49" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 35133 } } }
I20260619 06:50:03.029680 6452 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:03.029690 6451 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:03.032289 6459 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:50:03.036618 6459 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:50:03.045822 6459 catalog_manager.cc:1347] Generated new cluster ID: c464e65ddba6493483549b745b9ca112
I20260619 06:50:03.045949 6459 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:50:03.055924 6459 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:50:03.057297 6459 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:50:03.072906 6459 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6dbb40fa7854e3e88dd6d5fce368d49: Generated new TSK 0
I20260619 06:50:03.074195 6459 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:50:03.088410 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 6373
W20260619 06:50:03.089479 6435 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56221
W20260619 06:50:03.090122 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 6373: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:50:03.090268 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 6373
2026-06-19T06:50:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1929 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:50:03 dist-test-slave-5w2x krb5kdc[6471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:50:03 dist-test-slave-5w2x krb5kdc[6471](info): set up 2 sockets
Jun 19 06:50:03 dist-test-slave-5w2x krb5kdc[6471](info): commencing operation
krb5kdc: starting...
W20260619 06:50:04.212733 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:50:04 dist-test-slave-5w2x krb5kdc[6471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:50:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:50:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:50:04.473279 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:37637
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:35367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:37637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:04.810613 6484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:04.810950 6484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:04.811046 6484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:04.818498 6484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:04.818639 6484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:04.818687 6484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:04.818754 6484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:04.818796 6484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:04.829023 6484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:35367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:37637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:37637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:04.830510 6484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:04.831910 6484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:04.844537 6490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:04.844564 6492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:04.844623 6489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:04.845270 6484 server_base.cc:1034] running on GCE node
I20260619 06:50:04.846271 6484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:04.847836 6484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:04.849071 6484 hybrid_clock.cc:648] HybridClock initialized: now 1781851804848984 us; error 77 us; skew 500 ppm
I20260619 06:50:04.849495 6484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:04 dist-test-slave-5w2x krb5kdc[6471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851804, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:04.858536 6484 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:50:04.860316 6484 webserver.cc:460] Webserver started at http://127.16.79.126:45239/ using document root <none> and password file <none>
I20260619 06:50:04.861013 6484 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:04.861116 6484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:04.861398 6484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:04.864456 6484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "cd0f19038fc540f38de19107613b12c1"
format_stamp: "Formatted at 2026-06-19 06:50:04 on dist-test-slave-5w2x"
server_key: "73af39a31e39d0f5166c865d92c64104"
server_key_iv: "1de709c2fec06fbc1719d2383d3a8ba4"
server_key_version: "encryptionkey@0"
I20260619 06:50:04.865190 6484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "cd0f19038fc540f38de19107613b12c1"
format_stamp: "Formatted at 2026-06-19 06:50:04 on dist-test-slave-5w2x"
server_key: "73af39a31e39d0f5166c865d92c64104"
server_key_iv: "1de709c2fec06fbc1719d2383d3a8ba4"
server_key_version: "encryptionkey@0"
I20260619 06:50:04.870345 6484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:50:04.873834 6499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:04.875219 6484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 06:50:04.875361 6484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "cd0f19038fc540f38de19107613b12c1"
format_stamp: "Formatted at 2026-06-19 06:50:04 on dist-test-slave-5w2x"
server_key: "73af39a31e39d0f5166c865d92c64104"
server_key_iv: "1de709c2fec06fbc1719d2383d3a8ba4"
server_key_version: "encryptionkey@0"
I20260619 06:50:04.875535 6484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:04.881228 6484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:04.885500 6484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:04.885759 6484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:04.904444 6484 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:37637
I20260619 06:50:04.904479 6550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:37637 every 8 connection(s)
I20260619 06:50:04.905920 6484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:50:04.909000 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 6484
I20260619 06:50:04.909194 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:50:04.909792 16701 external_mini_cluster.cc:1428] Setting key 598513893413fadf3c46ac77b8ec6b2e
I20260619 06:50:04.910336 6551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:04.918462 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:04.911805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36383 (local address 127.16.79.126:37637)
0619 06:50:04.912659 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:04.912670 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:04.912711 (+ 41us) server_negotiation.cc:407] Connection header received
0619 06:50:04.914947 (+ 2236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:04.914968 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:04.915324 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:04.915718 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:04.916845 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:04.916855 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:04.916868 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:50:04.916881 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:04.916935 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:50:04.916958 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:04.917209 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:04.917358 (+ 149us) server_negotiation.cc:299] Negotiation successful
0619 06:50:04.917543 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":441,"threads_started":1}
I20260619 06:50:04.922443 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:50:04.924619 6551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1: Bootstrap starting.
I20260619 06:50:04.928706 6551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:04.930051 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:04.926872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42437 (local address 127.16.79.126:37637)
0619 06:50:04.927123 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:04.927130 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:04.928004 (+ 874us) server_negotiation.cc:407] Connection header received
0619 06:50:04.928265 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:04.928270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:04.928326 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:04.928430 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:04.929414 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:04.929419 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:04.929423 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 06:50:04.929442 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:04.929490 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 06:50:04.929505 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:04.929623 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:04.929715 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 06:50:04.929778 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 06:50:04.930014 6551 log.cc:824] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:04.933087 6551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1: No bootstrap required, opened a new log
I20260619 06:50:04.941303 6551 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f19038fc540f38de19107613b12c1" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37637 } }
I20260619 06:50:04.941725 6551 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 06:50:04.941841 6551 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0f19038fc540f38de19107613b12c1, State: Initialized, Role: FOLLOWER
I20260619 06:50:04.942487 6551 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f19038fc540f38de19107613b12c1" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37637 } }
I20260619 06:50:04.942727 6551 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 06:50:04.942884 6551 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 06:50:04.943058 6551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:04.945583 6551 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f19038fc540f38de19107613b12c1" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37637 } }
I20260619 06:50:04.946106 6551 leader_election.cc:302] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd0f19038fc540f38de19107613b12c1; no voters:
I20260619 06:50:04.946713 6551 leader_election.cc:288] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 06:50:04.946902 6561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:04.948014 6561 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [term 1 LEADER]: Becoming Leader. State: Replica: cd0f19038fc540f38de19107613b12c1, State: Running, Role: LEADER
I20260619 06:50:04.948616 6561 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f19038fc540f38de19107613b12c1" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37637 } }
I20260619 06:50:04.948720 6551 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:04.952354 6562 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd0f19038fc540f38de19107613b12c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f19038fc540f38de19107613b12c1" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37637 } } }
I20260619 06:50:04.952224 6563 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd0f19038fc540f38de19107613b12c1. Latest consensus state: current_term: 1 leader_uuid: "cd0f19038fc540f38de19107613b12c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0f19038fc540f38de19107613b12c1" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37637 } } }
I20260619 06:50:04.952778 6563 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:04.952778 6562 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:04.965251 6570 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:50:04.971186 6570 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:50:04.980973 6570 catalog_manager.cc:1347] Generated new cluster ID: e4e54df1230447659dc7289fd028d5e7
I20260619 06:50:04.981104 6570 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:50:04.994906 6570 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:50:04.995381 6570 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:50:05.001664 6570 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd0f19038fc540f38de19107613b12c1: Generated new TSK 0
I20260619 06:50:05.002640 6570 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:50:05.008181 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 6484
W20260619 06:50:05.009616 6546 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36383
W20260619 06:50:05.010325 16701 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 6484: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260619 06:50:05.010447 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 6484
2026-06-19T06:50:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1920 ms)
[----------] 18 tests from AuthTokenIssuingTest (35080 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:50:05 dist-test-slave-5w2x krb5kdc[6582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:50:05 dist-test-slave-5w2x krb5kdc[6582](info): set up 2 sockets
Jun 19 06:50:05 dist-test-slave-5w2x krb5kdc[6582](info): commencing operation
krb5kdc: starting...
W20260619 06:50:06.120838 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:50:06 dist-test-slave-5w2x krb5kdc[6582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:50:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:50:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:50:06.375133 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:40079
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:43201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:40079,127.16.79.125:38073,127.16.79.124:32935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:06.706660 6595 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 06:50:06.707053 6595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:06.707175 6595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:06.707317 6595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:06.715214 6595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:06.715332 6595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:06.715431 6595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:06.715517 6595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:06.715639 6595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:06.726179 6595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:43201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:40079,127.16.79.125:38073,127.16.79.124:32935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:40079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:06.727587 6595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:06.728967 6595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:06.740904 6601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:06.741037 6600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:06.741037 6603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:06.742431 6595 server_base.cc:1034] running on GCE node
I20260619 06:50:06.743306 6595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:06.744951 6595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:06.746181 6595 hybrid_clock.cc:648] HybridClock initialized: now 1781851806746099 us; error 42 us; skew 500 ppm
I20260619 06:50:06.746615 6595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:06 dist-test-slave-5w2x krb5kdc[6582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851806, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:06.749863 6595 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:50:06.751395 6595 webserver.cc:460] Webserver started at http://127.16.79.126:42369/ using document root <none> and password file <none>
I20260619 06:50:06.752086 6595 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:06.752264 6595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:06.752561 6595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:06.755508 6595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "18af0e07e24e464fb878e9c623337c11"
format_stamp: "Formatted at 2026-06-19 06:50:06 on dist-test-slave-5w2x"
server_key: "38e490f95c1ad2661bca073b1b02d3de"
server_key_iv: "f46e4da454577ef7dacdfbaed76de32f"
server_key_version: "encryptionkey@0"
I20260619 06:50:06.756325 6595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "18af0e07e24e464fb878e9c623337c11"
format_stamp: "Formatted at 2026-06-19 06:50:06 on dist-test-slave-5w2x"
server_key: "38e490f95c1ad2661bca073b1b02d3de"
server_key_iv: "f46e4da454577ef7dacdfbaed76de32f"
server_key_version: "encryptionkey@0"
I20260619 06:50:06.761545 6595 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 06:50:06.765116 6610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:06.766492 6595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260619 06:50:06.766685 6595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "18af0e07e24e464fb878e9c623337c11"
format_stamp: "Formatted at 2026-06-19 06:50:06 on dist-test-slave-5w2x"
server_key: "38e490f95c1ad2661bca073b1b02d3de"
server_key_iv: "f46e4da454577ef7dacdfbaed76de32f"
server_key_version: "encryptionkey@0"
I20260619 06:50:06.766927 6595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:06.787846 6595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:06.792588 6595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:06.792893 6595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:06.812734 6595 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:40079
I20260619 06:50:06.812774 6661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:40079 every 8 connection(s)
I20260619 06:50:06.814189 6595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:50:06.818267 6662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:06.820659 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 6595
I20260619 06:50:06.820890 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:50:06.821682 16701 external_mini_cluster.cc:1428] Setting key 12cebad37630f84c31e02d113128f9f4
I20260619 06:50:06.828332 6662 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:50:06 dist-test-slave-5w2x krb5kdc[6582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851806, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.79.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.125@KRBTEST.COM" created.
I20260619 06:50:06.857159 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:06.832898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:40079 (local address 127.0.0.1:36046)
0619 06:50:06.834043 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:06.834104 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:06.837012 (+ 2908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:06.837980 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:06.837989 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:06.838867 (+ 878us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:06.839400 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:06.839429 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:06.841399 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:06.841408 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:06.843414 (+ 2006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:06.843436 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:06.844554 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:06.844565 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:06.844834 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:06.845674 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:06.845717 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:06.848985 (+ 3268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:06.852636 (+ 3651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:06.852646 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:06.852656 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:06.853136 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:06.853682 (+ 546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:06.853689 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:06.853696 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:06.853893 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:06.855047 (+ 1154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:06.855053 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:06.855243 (+ 190us) client_negotiation.cc:769] Sending connection context
0619 06:50:06.855470 (+ 227us) client_negotiation.cc:240] Negotiation successful
0619 06:50:06.855899 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":543,"threads_started":1}
I20260619 06:50:06.857187 6666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:06.832898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36046 (local address 127.16.79.126:40079)
0619 06:50:06.834472 (+ 1574us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:06.834478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:06.834518 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:50:06.837473 (+ 2955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:06.837480 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:06.837725 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:06.837894 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:06.839641 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:06.841228 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:06.843595 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:06.844306 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:06.844449 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:06.849276 (+ 4827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:06.849284 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:06.849298 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:06.849336 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:06.852476 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:06.853301 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:06.853312 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:06.853315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:06.853531 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:06.854022 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:06.854029 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:06.854033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:06.854840 (+ 807us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:06.855069 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:06.855664 (+ 595us) server_negotiation.cc:299] Negotiation successful
0619 06:50:06.856118 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":116,"server-negotiator.queue_time_us":1396,"spinlock_wait_cycles":37632,"thread_start_us":543,"threads_started":1}
I20260619 06:50:06.860459 6662 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:50:06.861581 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:06.861171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:38073 (local address 127.0.0.1:56546)
0619 06:50:06.861347 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:06.861465 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260619 06:50:06.861884 6611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.79.125:38073: connect: Connection refused (error 111)
I20260619 06:50:06.863433 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:06.863080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:38073 (local address 127.0.0.1:56554)
0619 06:50:06.863244 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:06.863316 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 06:50:06.863626 6662 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.125:38073: Network error: Client connection negotiation failed: client connection to 127.16.79.125:38073: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.125.keytab.
Entry for principal kudu/127.16.79.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.125.keytab.
I20260619 06:50:06.884809 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.125:38073
--webserver_interface=127.16.79.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.125.keytab
--principal=kudu/127.16.79.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:43201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:40079,127.16.79.125:38073,127.16.79.124:32935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:50:06.912984 6662 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } attempt: 1
I20260619 06:50:06.914245 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:06.913718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:38073 (local address 127.0.0.1:56568)
0619 06:50:06.913936 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:06.914066 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260619 06:50:06.915238 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:06.914858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:38073 (local address 127.0.0.1:56572)
0619 06:50:06.914998 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:06.915086 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 06:50:06.915417 6662 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.125:38073: Network error: Client connection negotiation failed: client connection to 127.16.79.125:38073: connect: Connection refused (error 111)
I20260619 06:50:06.983701 6662 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } attempt: 2
I20260619 06:50:06.985034 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:06.984467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:38073 (local address 127.0.0.1:56574)
0619 06:50:06.984738 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:06.984850 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260619 06:50:06.986112 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:06.985687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:38073 (local address 127.0.0.1:56578)
0619 06:50:06.985836 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:06.985932 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 06:50:06.986315 6662 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.125:38073: Network error: Client connection negotiation failed: client connection to 127.16.79.125:38073: connect: Connection refused (error 111)
I20260619 06:50:07.077584 6662 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } attempt: 3
I20260619 06:50:07.078940 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.078342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:38073 (local address 127.0.0.1:56580)
0619 06:50:07.078604 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.078738 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260619 06:50:07.080174 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.079657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:38073 (local address 127.0.0.1:56596)
0619 06:50:07.079837 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.079940 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 06:50:07.080394 6662 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.125:38073: Network error: Client connection negotiation failed: client connection to 127.16.79.125:38073: connect: Connection refused (error 111)
W20260619 06:50:07.216995 6669 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 06:50:07.217324 6669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:07.217396 6669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:07.217476 6669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 06:50:07.223672 6662 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } attempt: 4
I20260619 06:50:07.224968 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.224377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:38073 (local address 127.0.0.1:56606)
0619 06:50:07.224689 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.224789 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
W20260619 06:50:07.225229 6669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:07.225325 6669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:07.225419 6669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:07.225503 6669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:07.225606 6669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:07.225987 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.225662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:38073 (local address 127.0.0.1:56608)
0619 06:50:07.225788 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.225873 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:38073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 06:50:07.226203 6662 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.125:38073: Network error: Client connection negotiation failed: client connection to 127.16.79.125:38073: connect: Connection refused (error 111)
I20260619 06:50:07.235620 6669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:43201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:40079,127.16.79.125:38073,127.16.79.124:32935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.125.keytab
--principal=kudu/127.16.79.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.125:38073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.79.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:07.236986 6669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:07.238294 6669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:07.250203 6677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:07.250350 6675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:07.250319 6674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:07.251396 6669 server_base.cc:1034] running on GCE node
I20260619 06:50:07.252283 6669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:07.253720 6669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:07.254966 6669 hybrid_clock.cc:648] HybridClock initialized: now 1781851807254897 us; error 41 us; skew 500 ppm
I20260619 06:50:07.255306 6669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:07 dist-test-slave-5w2x krb5kdc[6582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851807, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:07.259725 6669 init.cc:377] Logged in from keytab as kudu/127.16.79.125@KRBTEST.COM (short username kudu)
I20260619 06:50:07.261677 6669 webserver.cc:460] Webserver started at http://127.16.79.125:37037/ using document root <none> and password file <none>
I20260619 06:50:07.262482 6669 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:07.262604 6669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:07.262996 6669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:07.266685 6669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/data/instance:
uuid: "f6fdcbc163d240a1bccef30f9dd5c40e"
format_stamp: "Formatted at 2026-06-19 06:50:07 on dist-test-slave-5w2x"
server_key: "5ba02b152c4af85c51e39ed0589a62f9"
server_key_iv: "2db07966f5bc58d325aebfd15a29daee"
server_key_version: "encryptionkey@0"
I20260619 06:50:07.267585 6669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/wal/instance:
uuid: "f6fdcbc163d240a1bccef30f9dd5c40e"
format_stamp: "Formatted at 2026-06-19 06:50:07 on dist-test-slave-5w2x"
server_key: "5ba02b152c4af85c51e39ed0589a62f9"
server_key_iv: "2db07966f5bc58d325aebfd15a29daee"
server_key_version: "encryptionkey@0"
I20260619 06:50:07.273221 6669 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 06:50:07.276876 6684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:07.278270 6669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260619 06:50:07.278465 6669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/wal
uuid: "f6fdcbc163d240a1bccef30f9dd5c40e"
format_stamp: "Formatted at 2026-06-19 06:50:07 on dist-test-slave-5w2x"
server_key: "5ba02b152c4af85c51e39ed0589a62f9"
server_key_iv: "2db07966f5bc58d325aebfd15a29daee"
server_key_version: "encryptionkey@0"
I20260619 06:50:07.278698 6669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:07.301542 6669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:07.306360 6669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:07.306664 6669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:07.327675 6669 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.125:38073
I20260619 06:50:07.327718 6735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.125:38073 every 8 connection(s)
I20260619 06:50:07.329288 6669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/data/info.pb
I20260619 06:50:07.334471 6736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:07.339179 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 6669
I20260619 06:50:07.339389 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-1/wal/instance
I20260619 06:50:07.339872 16701 external_mini_cluster.cc:1428] Setting key 718a013f0660d2767bc9b4fa72b048d3
I20260619 06:50:07.346153 6736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.16.79.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.124@KRBTEST.COM" created.
Jun 19 06:50:07 dist-test-slave-5w2x krb5kdc[6582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851807, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.125@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:50:07.377820 6666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.350514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36060 (local address 127.16.79.126:40079)
0619 06:50:07.350791 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:07.350801 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:07.352251 (+ 1450us) server_negotiation.cc:407] Connection header received
0619 06:50:07.355817 (+ 3566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:07.355826 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:07.355933 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:07.356060 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:07.358598 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.359868 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.362773 (+ 2905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.363670 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.363786 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.369009 (+ 5223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:07.369019 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:07.369026 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:07.369079 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:07.373538 (+ 4459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.374640 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.374649 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.374655 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.374824 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.375363 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.375370 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.375375 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.375829 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:07.375996 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:07.377033 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0619 06:50:07.377339 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260619 06:50:07.378969 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.351005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:40079 (local address 127.0.0.1:36060)
0619 06:50:07.352007 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.352070 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:07.355373 (+ 3303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:07.356427 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:07.356441 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:07.357355 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:07.358324 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:07.358358 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.360085 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.360093 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:07.362554 (+ 2461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:07.362576 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.363948 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.363958 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:07.364392 (+ 434us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.365259 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:07.365297 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:07.368699 (+ 3402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:07.373745 (+ 5046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:07.373756 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:07.373780 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:07.374468 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:07.374958 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:07.374966 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:07.374973 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:07.375230 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:07.375975 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:07.375986 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:07.376532 (+ 546us) client_negotiation.cc:769] Sending connection context
0619 06:50:07.376854 (+ 322us) client_negotiation.cc:240] Negotiation successful
0619 06:50:07.377440 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":369,"threads_started":1}
I20260619 06:50:07.381027 6736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:50:07 dist-test-slave-5w2x krb5kdc[6582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851807, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.125@KRBTEST.COM for kudu/127.16.79.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.124.keytab.
Entry for principal kudu/127.16.79.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.124.keytab.
I20260619 06:50:07.400286 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.124:32935
--webserver_interface=127.16.79.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.124.keytab
--principal=kudu/127.16.79.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:43201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:40079,127.16.79.125:38073,127.16.79.124:32935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:50:07.402137 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.381957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:38073 (local address 127.0.0.1:56610)
0619 06:50:07.382184 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.382213 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:07.382344 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:07.383801 (+ 1457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:07.383808 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:07.384436 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:07.384650 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:07.384664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.387011 (+ 2347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.387020 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:07.388659 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:07.388678 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.390109 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.390117 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:07.390343 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.391067 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:07.391097 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:07.394670 (+ 3573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:07.398862 (+ 4192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:07.398869 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:07.398873 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:07.399292 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:07.399841 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:07.399849 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:07.399855 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:07.400032 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:07.400910 (+ 878us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:07.400918 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:07.401122 (+ 204us) client_negotiation.cc:769] Sending connection context
0619 06:50:07.401278 (+ 156us) client_negotiation.cc:240] Negotiation successful
0619 06:50:07.401620 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 06:50:07.402341 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.382369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56610 (local address 127.16.79.125:38073)
0619 06:50:07.383149 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:07.383160 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:07.383193 (+ 33us) server_negotiation.cc:407] Connection header received
0619 06:50:07.383257 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:07.383268 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:07.383547 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:07.383780 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:07.384855 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.386845 (+ 1990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.388817 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.389906 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.390055 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.394922 (+ 4867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:07.394944 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:07.394967 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:07.395025 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:07.398724 (+ 3699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.399492 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.399506 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.399512 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.399700 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.400185 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.400191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.400197 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.400757 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:07.401004 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:07.401493 (+ 489us) server_negotiation.cc:299] Negotiation successful
0619 06:50:07.401814 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":412,"threads_started":1}
I20260619 06:50:07.405623 6736 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:50:07.407171 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.406529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58304)
0619 06:50:07.406819 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.406945 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260619 06:50:07.407526 6688 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
I20260619 06:50:07.409509 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.409025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58316)
0619 06:50:07.409248 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.409351 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
W20260619 06:50:07.409860 6736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:32935: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
I20260619 06:50:07.459266 6736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } attempt: 1
I20260619 06:50:07.460711 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.460062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58328)
0619 06:50:07.460334 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.460476 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260619 06:50:07.462451 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.461579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58336)
0619 06:50:07.462007 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.462183 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215}
W20260619 06:50:07.462764 6736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:32935: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
I20260619 06:50:07.525871 6662 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } attempt: 5
I20260619 06:50:07.531054 6736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } attempt: 2
I20260619 06:50:07.532469 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.531781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58348)
0619 06:50:07.532097 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.532244 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
Jun 19 06:50:07 dist-test-slave-5w2x krb5kdc[6582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851806, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for kudu/127.16.79.125@KRBTEST.COM
I20260619 06:50:07.538173 6739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.533257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58358)
0619 06:50:07.537826 (+ 4569us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.537942 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4433,"threads_started":1}
W20260619 06:50:07.543525 6736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:32935: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
I20260619 06:50:07.544453 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.526738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:38073 (local address 127.0.0.1:56622)
0619 06:50:07.527120 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.527148 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:07.527324 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:07.527806 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:07.527811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:07.528355 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:07.528655 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:07.528669 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.530267 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.530273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:07.531673 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:07.531690 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.533045 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.533052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:07.533281 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.533976 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:07.534000 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:07.536968 (+ 2968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:07.541295 (+ 4327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:07.541300 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:07.541305 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:07.541843 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:07.542263 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:07.542268 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:07.542271 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:07.542415 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:07.543289 (+ 874us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:07.543295 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:07.543488 (+ 193us) client_negotiation.cc:769] Sending connection context
0619 06:50:07.543735 (+ 247us) client_negotiation.cc:240] Negotiation successful
0619 06:50:07.544036 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205}
I20260619 06:50:07.545051 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.526854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56622 (local address 127.16.79.125:38073)
0619 06:50:07.527125 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:07.527133 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:07.527255 (+ 122us) server_negotiation.cc:407] Connection header received
0619 06:50:07.527478 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:07.527487 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:07.527599 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:07.527727 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:07.528845 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.530105 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.531868 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.532718 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.532841 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.537194 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:07.537200 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:07.537205 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:07.537245 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:07.541131 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.541986 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.541992 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.541995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.542133 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.542555 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.542559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.542563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.543054 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:07.543216 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:07.544360 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0619 06:50:07.544622 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260619 06:50:07.546185 6662 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:50:07.547468 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.546948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58364)
0619 06:50:07.547139 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.547250 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260619 06:50:07.548623 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.548166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58366)
0619 06:50:07.548341 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.548448 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 06:50:07.548919 6662 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:32935: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
I20260619 06:50:07.600208 6662 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } attempt: 1
I20260619 06:50:07.601641 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.601155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58376)
0619 06:50:07.601367 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.601458 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260619 06:50:07.602645 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.602313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58378)
0619 06:50:07.602459 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.602534 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 06:50:07.602857 6662 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:32935: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
I20260619 06:50:07.634802 6736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } attempt: 3
I20260619 06:50:07.636256 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.635593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58382)
0619 06:50:07.635819 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.636014 (+ 195us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 06:50:07.637300 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.636907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58386)
0619 06:50:07.637136 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.637199 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260619 06:50:07.637495 6736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:32935: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
I20260619 06:50:07.671066 6662 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } attempt: 2
I20260619 06:50:07.672271 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.671742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58392)
0619 06:50:07.671956 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.672061 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 06:50:07.673285 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.672909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58398)
0619 06:50:07.673107 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.673171 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 06:50:07.673525 6662 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:32935: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
W20260619 06:50:07.745620 6743 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 06:50:07.746040 6743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:07.746163 6743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:07.746320 6743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:07.753830 6743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:07.753928 6743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:07.753991 6743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:07.754041 6743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:07.754096 6743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:07.764037 6743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:43201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:40079,127.16.79.125:38073,127.16.79.124:32935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.124.keytab
--principal=kudu/127.16.79.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.124:32935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.79.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:07.765470 6743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:07.766711 6743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:07.778514 6752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:07.778622 6749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:07.778584 6750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:07.779455 6743 server_base.cc:1034] running on GCE node
I20260619 06:50:07.779743 6662 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } attempt: 3
I20260619 06:50:07.780376 6743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:07.780699 6736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } attempt: 4
I20260619 06:50:07.780860 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.780343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58414)
0619 06:50:07.780554 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.780667 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260619 06:50:07.781702 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.781231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58416)
0619 06:50:07.781428 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.781522 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260619 06:50:07.781978 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.781529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58432)
0619 06:50:07.781722 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.781836 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260619 06:50:07.782245 6662 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:32935: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
I20260619 06:50:07.782400 6743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:07.782948 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.782582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58444)
0619 06:50:07.782741 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.782815 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 06:50:07.783161 6736 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:32935: Network error: Client connection negotiation failed: client connection to 127.16.79.124:32935: connect: Connection refused (error 111)
I20260619 06:50:07.783684 6743 hybrid_clock.cc:648] HybridClock initialized: now 1781851807783607 us; error 53 us; skew 500 ppm
I20260619 06:50:07.784016 6743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:07 dist-test-slave-5w2x krb5kdc[6582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851807, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:07.787012 6743 init.cc:377] Logged in from keytab as kudu/127.16.79.124@KRBTEST.COM (short username kudu)
I20260619 06:50:07.788540 6743 webserver.cc:460] Webserver started at http://127.16.79.124:41667/ using document root <none> and password file <none>
I20260619 06:50:07.789201 6743 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:07.789330 6743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:07.789631 6743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:07.792546 6743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/data/instance:
uuid: "c61e661d2c754c73b6ed27b748b00af8"
format_stamp: "Formatted at 2026-06-19 06:50:07 on dist-test-slave-5w2x"
server_key: "945c848e696aa74ee8fdceb606c62da4"
server_key_iv: "842995a031e2f339de5955592af02ce0"
server_key_version: "encryptionkey@0"
I20260619 06:50:07.793345 6743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/wal/instance:
uuid: "c61e661d2c754c73b6ed27b748b00af8"
format_stamp: "Formatted at 2026-06-19 06:50:07 on dist-test-slave-5w2x"
server_key: "945c848e696aa74ee8fdceb606c62da4"
server_key_iv: "842995a031e2f339de5955592af02ce0"
server_key_version: "encryptionkey@0"
I20260619 06:50:07.798516 6743 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:50:07.802201 6759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:07.803582 6743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:50:07.803781 6743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/wal
uuid: "c61e661d2c754c73b6ed27b748b00af8"
format_stamp: "Formatted at 2026-06-19 06:50:07 on dist-test-slave-5w2x"
server_key: "945c848e696aa74ee8fdceb606c62da4"
server_key_iv: "842995a031e2f339de5955592af02ce0"
server_key_version: "encryptionkey@0"
I20260619 06:50:07.804016 6743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:07.823271 6743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:07.827389 6743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:07.827661 6743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:07.846084 6743 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.124:32935
I20260619 06:50:07.846103 6810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.124:32935 every 8 connection(s)
I20260619 06:50:07.847497 6743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/data/info.pb
I20260619 06:50:07.848759 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 6743
I20260619 06:50:07.848941 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781851451399607-16701-0/minicluster-data/master-2/wal/instance
I20260619 06:50:07.849439 16701 external_mini_cluster.cc:1428] Setting key be76aea443408d64c2d7e49c2cec078e
I20260619 06:50:07.852231 6811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:07.858796 6811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:50:07 dist-test-slave-5w2x krb5kdc[6582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.124@KRBTEST.COM
Jun 19 06:50:07 dist-test-slave-5w2x krb5kdc[6582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851807, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.124@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:50:07.872289 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.851481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58460 (local address 127.16.79.124:32935)
0619 06:50:07.852349 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:07.852368 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:07.852421 (+ 53us) server_negotiation.cc:407] Connection header received
0619 06:50:07.853973 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:07.854000 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:07.854371 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:07.854783 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:07.856260 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.857594 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.858853 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.859563 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.859793 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.864330 (+ 4537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:07.864339 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:07.864352 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:07.864396 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:07.867947 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.868853 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.868860 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.868865 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.869034 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.869494 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.869498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.869500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.870100 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:07.870460 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:07.870819 (+ 359us) server_negotiation.cc:299] Negotiation successful
0619 06:50:07.871228 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":337,"threads_started":1}
I20260619 06:50:07.877216 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.861598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:40079 (local address 127.0.0.1:36062)
0619 06:50:07.862406 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.862454 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:07.862617 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:07.863024 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:07.863036 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:07.863681 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:07.863913 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:07.863922 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.865373 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.865377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:07.866509 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:07.866520 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.867486 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.867491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:07.867613 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.868248 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:07.868274 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:07.870661 (+ 2387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:07.874157 (+ 3496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:07.874164 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:07.874169 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:07.874760 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:07.875236 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:07.875240 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:07.875243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:07.875388 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:07.876120 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:07.876153 (+ 33us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:07.876323 (+ 170us) client_negotiation.cc:769] Sending connection context
0619 06:50:07.876500 (+ 177us) client_negotiation.cc:240] Negotiation successful
0619 06:50:07.876796 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":400,"threads_started":1}
I20260619 06:50:07.877295 6666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.861625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36062 (local address 127.16.79.126:40079)
0619 06:50:07.861855 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:07.861864 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:07.862559 (+ 695us) server_negotiation.cc:407] Connection header received
0619 06:50:07.862740 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:07.862744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:07.862806 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:07.862919 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:07.864071 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.865204 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.866652 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.867302 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.867439 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.870886 (+ 3447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:07.870913 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:07.870918 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:07.870959 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:07.874020 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.874888 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.874896 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.874901 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.875112 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.875502 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.875510 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.875517 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.875891 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:07.876076 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:07.876642 (+ 566us) server_negotiation.cc:299] Negotiation successful
0619 06:50:07.876914 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":77}
I20260619 06:50:07.879036 6811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:50:07 dist-test-slave-5w2x krb5kdc[6582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
Jun 19 06:50:07 dist-test-slave-5w2x krb5kdc[6582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851807, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.124@KRBTEST.COM for kudu/127.16.79.125@KRBTEST.COM
I20260619 06:50:07.893302 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.875711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36070 (local address 127.16.79.126:40079)
0619 06:50:07.876306 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:07.876310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:07.876330 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:50:07.876367 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:07.876370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:07.876419 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:07.876533 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:07.878483 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.879460 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.881605 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.882326 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.882449 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.887583 (+ 5134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:07.887588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:07.887591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:07.887626 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:07.890268 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.891098 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.891103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.891107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.891276 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.891633 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.891639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.891644 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.891988 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:07.892305 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:07.892613 (+ 308us) server_negotiation.cc:299] Negotiation successful
0619 06:50:07.892859 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":470,"thread_start_us":242,"threads_started":1}
I20260619 06:50:07.893833 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.879860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56630 (local address 127.16.79.125:38073)
0619 06:50:07.880032 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:07.880037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:07.880249 (+ 212us) server_negotiation.cc:407] Connection header received
0619 06:50:07.880427 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:07.880437 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:07.880496 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:07.880605 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:07.881264 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.882279 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.883827 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.884319 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.884412 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.887858 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:07.887862 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:07.887866 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:07.887896 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:07.890948 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.891738 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.891741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.891743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.891876 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.892297 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.892301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.892304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.892658 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:07.892810 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:07.893285 (+ 475us) server_negotiation.cc:299] Negotiation successful
0619 06:50:07.893531 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:50:07.894234 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.879861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:38073 (local address 127.0.0.1:56630)
0619 06:50:07.880111 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.880125 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:07.880299 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:07.880621 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:07.880625 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:07.880978 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:07.881122 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:07.881130 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.882444 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.882452 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:07.883686 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:07.883701 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.884472 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.884480 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:07.884705 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.885228 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:07.885255 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:07.887674 (+ 2419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:07.891104 (+ 3430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:07.891109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:07.891112 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:07.891618 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:07.891986 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:07.891991 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:07.891994 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:07.892194 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:07.892787 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:07.892793 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:07.892966 (+ 173us) client_negotiation.cc:769] Sending connection context
0619 06:50:07.893168 (+ 202us) client_negotiation.cc:240] Negotiation successful
0619 06:50:07.893874 (+ 706us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260619 06:50:07.895241 6811 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:50:07 dist-test-slave-5w2x krb5kdc[6582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851807, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.124@KRBTEST.COM for kudu/127.16.79.124@KRBTEST.COM
Jun 19 06:50:07 dist-test-slave-5w2x krb5kdc[6582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851806, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.125@KRBTEST.COM
I20260619 06:50:07.910882 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.896380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58466 (local address 127.16.79.124:32935)
0619 06:50:07.896530 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:07.896535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:07.896548 (+ 13us) server_negotiation.cc:407] Connection header received
0619 06:50:07.896585 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:07.896588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:07.896635 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:07.896706 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:07.897532 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.898558 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.900182 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.900802 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.900917 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.904592 (+ 3675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:07.904597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:07.904601 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:07.904655 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:07.907958 (+ 3303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.908810 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.908816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.908819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.908970 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.909345 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.909350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.909353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.909726 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:07.909871 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:07.910210 (+ 339us) server_negotiation.cc:299] Negotiation successful
0619 06:50:07.910463 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":47}
I20260619 06:50:07.910953 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.896027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58466)
0619 06:50:07.896234 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.896253 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:07.896355 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:07.896768 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:07.896776 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:07.897262 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:07.897405 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:07.897413 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.898675 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.898680 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:07.900018 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:07.900032 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.900962 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.900968 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:07.901197 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.901643 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:07.901664 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:07.904392 (+ 2728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:07.908109 (+ 3717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:07.908113 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:07.908115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:07.908678 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:07.909090 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:07.909097 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:07.909101 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:07.909239 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:07.909848 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:07.909852 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:07.909994 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 06:50:07.910132 (+ 138us) client_negotiation.cc:240] Negotiation successful
0619 06:50:07.910601 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":171}
I20260619 06:50:07.911334 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.895115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56642 (local address 127.16.79.125:38073)
0619 06:50:07.895260 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:07.895264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:07.895418 (+ 154us) server_negotiation.cc:407] Connection header received
0619 06:50:07.895559 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:07.895562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:07.895623 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:07.895730 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:07.897248 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.898307 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.899997 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.900628 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.900754 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.905576 (+ 4822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:07.905584 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:07.905591 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:07.905635 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:07.908827 (+ 3192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.909432 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.909437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.909441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.909584 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.909926 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.909931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.909935 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.910207 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:07.910465 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:07.910755 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 06:50:07.910980 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 06:50:07.916107 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:50:07.916765 6811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8: Bootstrap starting.
I20260619 06:50:07.920460 6811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:07.921763 6811 log.cc:824] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:07.918817 6637 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "18af0e07e24e464fb878e9c623337c11"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36070
I20260619 06:50:07.924888 6811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8: No bootstrap required, opened a new log
I20260619 06:50:07.931758 6811 raft_consensus.cc:348] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } }
I20260619 06:50:07.932093 6811 raft_consensus.cc:729] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c61e661d2c754c73b6ed27b748b00af8, State: Initialized, Role: FOLLOWER
I20260619 06:50:07.932557 6811 consensus_queue.cc:260] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } }
I20260619 06:50:07.933483 6818 sys_catalog.cc:455] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } } }
I20260619 06:50:07.933887 6818 sys_catalog.cc:458] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:50:07.934459 6811 sys_catalog.cc:564] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 06:50:07.944286 6829 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:50:07.944425 6829 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:50:07.941988 6743 master_runner.cc:186] Error getting master registration for 127.16.79.126:40079: OK, instance_id { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" instance_seqno: 1781851806806591 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 06:50:07.952427 6743 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 06:50:07.959501 6662 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } attempt: 4
Jun 19 06:50:07 dist-test-slave-5w2x krb5kdc[6582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851806, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for kudu/127.16.79.124@KRBTEST.COM
I20260619 06:50:07.976325 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.960411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58476)
0619 06:50:07.960608 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:07.960632 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:07.960762 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:07.961139 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:07.961146 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:07.961697 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:07.961912 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:07.961925 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.963405 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.963411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:07.965048 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:07.965065 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.966102 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.966109 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:07.966310 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.967029 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:07.967067 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:07.969927 (+ 2860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:07.973786 (+ 3859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:07.973792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:07.973796 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:07.974209 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:07.974587 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:07.974593 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:07.974596 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:07.974754 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:07.975311 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:07.975316 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:07.975471 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 06:50:07.975615 (+ 144us) client_negotiation.cc:240] Negotiation successful
0619 06:50:07.975904 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 06:50:07.976330 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:07.960485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58476 (local address 127.16.79.124:32935)
0619 06:50:07.960724 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:07.960730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:07.960751 (+ 21us) server_negotiation.cc:407] Connection header received
0619 06:50:07.960885 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:07.960890 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:07.960968 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:07.961067 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:07.962074 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.963247 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.965205 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:07.965933 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:07.966057 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:07.970165 (+ 4108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:07.970171 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:07.970175 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:07.970208 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:07.973638 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.974334 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.974340 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.974344 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.974467 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:07.974870 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:07.974875 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:07.974878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:07.975186 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:07.975367 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:07.975726 (+ 359us) server_negotiation.cc:299] Negotiation successful
0619 06:50:07.975943 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 06:50:07.981150 6662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11: Bootstrap starting.
I20260619 06:50:07.984997 6662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:07.986073 6662 log.cc:824] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:07.988885 6662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11: No bootstrap required, opened a new log
I20260619 06:50:07.996556 6662 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } }
I20260619 06:50:07.996969 6662 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18af0e07e24e464fb878e9c623337c11, State: Initialized, Role: FOLLOWER
I20260619 06:50:07.997450 6662 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } }
I20260619 06:50:07.998415 6832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } } }
I20260619 06:50:07.998832 6832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:50:07.999693 6662 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:07.999774 6637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 06:50:07.999958 6637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:08.004063 6637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } }
I20260619 06:50:08.006498 6637 leader_election.cc:288] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [CANDIDATE]: Term 1 election: Requested vote from peers f6fdcbc163d240a1bccef30f9dd5c40e (127.16.79.125:38073), c61e661d2c754c73b6ed27b748b00af8 (127.16.79.124:32935)
I20260619 06:50:08.007253 6786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "18af0e07e24e464fb878e9c623337c11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c61e661d2c754c73b6ed27b748b00af8"
I20260619 06:50:08.007843 6786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:08.012419 6786 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18af0e07e24e464fb878e9c623337c11 in term 1.
I20260619 06:50:08.013289 6614 leader_election.cc:302] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18af0e07e24e464fb878e9c623337c11, c61e661d2c754c73b6ed27b748b00af8; no voters:
I20260619 06:50:08.013916 6832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:08.017987 6832 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [term 1 LEADER]: Becoming Leader. State: Replica: 18af0e07e24e464fb878e9c623337c11, State: Running, Role: LEADER
I20260619 06:50:08.018669 6832 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } }
I20260619 06:50:08.022238 6844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18af0e07e24e464fb878e9c623337c11. Latest consensus state: current_term: 1 leader_uuid: "18af0e07e24e464fb878e9c623337c11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } } }
I20260619 06:50:08.022682 6844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:08.029793 6849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260619 06:50:08.045780 6616 tablet.cc:2369] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:50:08.046039 6849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 06:50:08.048362 6850 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 06:50:08.068169 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:08.024373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36076 (local address 127.16.79.126:40079)
0619 06:50:08.024573 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.024581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.041353 (+ 16772us) server_negotiation.cc:407] Connection header received
0619 06:50:08.042839 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.042845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.042932 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.043007 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:08.045410 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.046495 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.051036 (+ 4541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.051751 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.051853 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:08.058415 (+ 6562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:08.058421 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:08.058427 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:08.058466 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:08.062857 (+ 4391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:08.064286 (+ 1429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:08.064323 (+ 37us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:08.064327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:08.064601 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:08.065415 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:08.065420 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:08.065423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:08.065808 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:08.066084 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:08.067019 (+ 935us) server_negotiation.cc:299] Negotiation successful
0619 06:50:08.067426 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 06:50:08.059644 6595 master_runner.cc:186] Error getting master registration for 127.16.79.125:38073: OK, instance_id { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" instance_seqno: 1781851807321613 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260619 06:50:08.079779 6765 tablet.cc:2369] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:50:08.081876 6595 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 06:50:08.082342 6736 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } attempt: 5
Jun 19 06:50:08 dist-test-slave-5w2x krb5kdc[6582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851807, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.125@KRBTEST.COM for kudu/127.16.79.124@KRBTEST.COM
I20260619 06:50:08.100867 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:08.024545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56656 (local address 127.16.79.125:38073)
0619 06:50:08.024718 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.024726 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.041950 (+ 17224us) server_negotiation.cc:407] Connection header received
0619 06:50:08.043530 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.043538 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.043637 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.043722 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:08.070740 (+ 27018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.071924 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.074026 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.074674 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.074798 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:08.092293 (+ 17495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:08.092303 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:08.092308 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:08.092360 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:08.096572 (+ 4212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:08.098289 (+ 1717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:08.098297 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:08.098304 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:08.098462 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:08.098955 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:08.098961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:08.098965 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:08.099336 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:08.099583 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:08.100240 (+ 657us) server_negotiation.cc:299] Negotiation successful
0619 06:50:08.100476 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 06:50:08.107136 6786 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8 [term 1 FOLLOWER]: Refusing update from remote peer 18af0e07e24e464fb878e9c623337c11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:08.109560 6844 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:08.110112 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:08.029245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58480 (local address 127.16.79.124:32935)
0619 06:50:08.029427 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.029433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.042588 (+ 13155us) server_negotiation.cc:407] Connection header received
0619 06:50:08.043053 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.043062 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.043181 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.043266 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:08.070987 (+ 27721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.072173 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.075676 (+ 3503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.076379 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.076502 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:08.097589 (+ 21087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:08.097597 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:08.097603 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:08.097643 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:08.101489 (+ 3846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:08.104332 (+ 2843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:08.104337 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:08.104340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:08.104588 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:08.106276 (+ 1688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:08.106316 (+ 40us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:08.106320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:08.106931 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:08.107406 (+ 475us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:08.108539 (+ 1133us) server_negotiation.cc:299] Negotiation successful
0619 06:50:08.109205 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 06:50:08.122252 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:08.082939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:32935 (local address 127.0.0.1:58486)
0619 06:50:08.083108 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:08.083127 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:08.083265 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:08.084975 (+ 1710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:08.084980 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:08.085459 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:08.085684 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:08.085697 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.088604 (+ 2907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.088612 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:08.089990 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:08.090006 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.091492 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.091498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:08.091694 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:08.092427 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:08.092447 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:08.095961 (+ 3514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:08.117816 (+ 21855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:08.117826 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:08.117832 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:08.118491 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:08.119538 (+ 1047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:08.119547 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:08.119553 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:08.119782 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:08.121173 (+ 1391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:08.121182 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:08.121373 (+ 191us) client_negotiation.cc:769] Sending connection context
0619 06:50:08.121535 (+ 162us) client_negotiation.cc:240] Negotiation successful
0619 06:50:08.121839 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 06:50:08.124284 6856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:08.083289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58486 (local address 127.16.79.124:32935)
0619 06:50:08.084381 (+ 1092us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.084390 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.084448 (+ 58us) server_negotiation.cc:407] Connection header received
0619 06:50:08.084528 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.084534 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.084712 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.085879 (+ 1167us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:08.086309 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.088391 (+ 2082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.090221 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.091268 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.092640 (+ 1372us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:08.096252 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:08.096262 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:08.096269 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:08.096338 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:08.117508 (+ 21170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:08.118957 (+ 1449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:08.118965 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:08.118969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:08.119316 (+ 347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:08.119990 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:08.119997 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:08.120003 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:08.120920 (+ 917us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:08.122806 (+ 1886us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:08.123038 (+ 232us) server_negotiation.cc:299] Negotiation successful
0619 06:50:08.123394 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":875,"thread_start_us":488,"threads_started":1}
I20260619 06:50:08.130792 6736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e: Bootstrap starting.
I20260619 06:50:08.134090 6818 sys_catalog.cc:455] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18af0e07e24e464fb878e9c623337c11. Latest consensus state: current_term: 1 leader_uuid: "18af0e07e24e464fb878e9c623337c11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } } }
I20260619 06:50:08.134428 6818 sys_catalog.cc:458] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:50:08.136437 6736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:08.138118 6736 log.cc:824] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:08.144078 6855 mvcc.cc:205] Tried to move back new op lower bound from 7298465005994827776 to 7298465005650632704. Current Snapshot: MvccSnapshot[applied={T|T < 7298465005994827776}]
I20260619 06:50:08.159870 6857 mvcc.cc:205] Tried to move back new op lower bound from 7298465005994827776 to 7298465005650632704. Current Snapshot: MvccSnapshot[applied={T|T < 7298465005994827776}]
I20260619 06:50:08.181571 6818 sys_catalog.cc:455] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "18af0e07e24e464fb878e9c623337c11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } } }
I20260619 06:50:08.182015 6818 sys_catalog.cc:458] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:50:08.194038 6832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "18af0e07e24e464fb878e9c623337c11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } } }
I20260619 06:50:08.195331 6832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:08.195679 6849 catalog_manager.cc:1347] Generated new cluster ID: 196b4d26647d4863b76d7c4f87b4fadf
I20260619 06:50:08.195781 6849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 06:50:08.196772 6736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e: No bootstrap required, opened a new log
I20260619 06:50:08.208437 6736 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } }
I20260619 06:50:08.208947 6736 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6fdcbc163d240a1bccef30f9dd5c40e, State: Initialized, Role: FOLLOWER
I20260619 06:50:08.209537 6736 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } }
I20260619 06:50:08.210793 6862 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } } }
I20260619 06:50:08.211544 6862 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:50:08.212116 6736 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:08.212507 6710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:08.217176 6710 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e [term 1 FOLLOWER]: Refusing update from remote peer 18af0e07e24e464fb878e9c623337c11: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:08.218113 6832 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:08.221654 6711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "18af0e07e24e464fb878e9c623337c11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e"
I20260619 06:50:08.231477 6849 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:50:08.233951 6849 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:50:08.237654 6711 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18af0e07e24e464fb878e9c623337c11 in term 1.
I20260619 06:50:08.285699 6867 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "18af0e07e24e464fb878e9c623337c11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } } }
I20260619 06:50:08.286099 6867 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:50:08.286185 6862 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18af0e07e24e464fb878e9c623337c11. Latest consensus state: current_term: 1 leader_uuid: "18af0e07e24e464fb878e9c623337c11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } } }
I20260619 06:50:08.286566 6862 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:50:08.288004 6844 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "18af0e07e24e464fb878e9c623337c11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "18af0e07e24e464fb878e9c623337c11" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 40079 } } peers { permanent_uuid: "f6fdcbc163d240a1bccef30f9dd5c40e" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 38073 } } peers { permanent_uuid: "c61e661d2c754c73b6ed27b748b00af8" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 32935 } } }
I20260619 06:50:08.288359 6844 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:08.299997 6849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18af0e07e24e464fb878e9c623337c11: Generated new TSK 0
I20260619 06:50:08.301656 6849 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:50:08.330053 6877 catalog_manager.cc:1259] Loaded cluster ID: 196b4d26647d4863b76d7c4f87b4fadf
I20260619 06:50:08.330276 6877 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e: loading cluster ID for follower catalog manager: success
I20260619 06:50:08.342690 6877 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e: acquiring CA information for follower catalog manager: success
I20260619 06:50:08.345430 6877 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f6fdcbc163d240a1bccef30f9dd5c40e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 06:50:08.482103 6816 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:08.454361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36080 (local address 127.16.79.126:40079)
0619 06:50:08.454568 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.454575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.479829 (+ 25254us) server_negotiation.cc:407] Connection header received
0619 06:50:08.480425 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.480435 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.480588 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.480666 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:08.481833 (+ 1167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36080: BlockingRecv error: recv got EOF from 127.0.0.1:36080 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260619 06:50:08.485415 6856 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:08.474281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58488 (local address 127.16.79.124:32935)
0619 06:50:08.474514 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.474524 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.483551 (+ 9027us) server_negotiation.cc:407] Connection header received
0619 06:50:08.483992 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.483999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.484153 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.484225 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:08.485179 (+ 954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58488: BlockingRecv error: recv got EOF from 127.0.0.1:58488 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260619 06:50:08.487538 6741 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:08.460656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56660 (local address 127.16.79.125:38073)
0619 06:50:08.460867 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.460873 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.483042 (+ 22169us) server_negotiation.cc:407] Connection header received
0619 06:50:08.485812 (+ 2770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.485821 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.485945 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.486029 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:08.487309 (+ 1280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56660: BlockingRecv error: recv got EOF from 127.0.0.1:56660 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 06:50:08.540604 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:08.520899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56670 (local address 127.16.79.125:38073)
0619 06:50:08.521145 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.521154 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.532783 (+ 11629us) server_negotiation.cc:407] Connection header received
0619 06:50:08.533078 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.533084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.533302 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.533516 (+ 214us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:08.534230 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.535391 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.537469 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.538161 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.538264 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:08.539046 (+ 782us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:08.539762 (+ 716us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:08.539887 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:08.540129 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 06:50:08.540257 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260619 06:50:08.582733 6856 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:08.574366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58496 (local address 127.16.79.124:32935)
0619 06:50:08.574933 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.574940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.580818 (+ 5878us) server_negotiation.cc:407] Connection header received
0619 06:50:08.581064 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.581070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.581208 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.581325 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:08.582425 (+ 1100us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58496: BlockingRecv error: recv got EOF from 127.0.0.1:58496 (error 108)
Metrics: {"server-negotiator.queue_time_us":438}
I20260619 06:50:08.595468 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:08.588350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56680 (local address 127.16.79.125:38073)
0619 06:50:08.588533 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.588540 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.588562 (+ 22us) server_negotiation.cc:407] Connection header received
0619 06:50:08.588807 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.588813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.588937 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.589000 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:08.589637 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.590825 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.593005 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.593741 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.593847 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:08.594513 (+ 666us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:08.594708 (+ 195us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:08.594784 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:08.595017 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 06:50:08.595116 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260619 06:50:08.606604 6856 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:08.604488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58498 (local address 127.16.79.124:32935)
0619 06:50:08.604701 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.604707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.605106 (+ 399us) server_negotiation.cc:407] Connection header received
0619 06:50:08.605296 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.605302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.605403 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.605522 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:08.606367 (+ 845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58498: BlockingRecv error: recv got EOF from 127.0.0.1:58498 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 06:50:08.624120 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:08.611789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56692 (local address 127.16.79.125:38073)
0619 06:50:08.611970 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.611977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.616816 (+ 4839us) server_negotiation.cc:407] Connection header received
0619 06:50:08.617054 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.617061 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.617162 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.617276 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:08.617986 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.619301 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.621578 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.622319 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.622445 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:08.623106 (+ 661us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:08.623279 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:08.623351 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:08.623622 (+ 271us) server_negotiation.cc:299] Negotiation successful
0619 06:50:08.623731 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260619 06:50:08.634343 6856 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:08.631683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58506 (local address 127.16.79.124:32935)
0619 06:50:08.631878 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.631885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.632490 (+ 605us) server_negotiation.cc:407] Connection header received
0619 06:50:08.632795 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.632801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.632906 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.632988 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:08.634082 (+ 1094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58506: BlockingRecv error: recv got EOF from 127.0.0.1:58506 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 06:50:08.652856 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:08.646209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56704 (local address 127.16.79.125:38073)
0619 06:50:08.646436 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.646443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.646997 (+ 554us) server_negotiation.cc:407] Connection header received
0619 06:50:08.647223 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.647229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.647329 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.647430 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:08.648013 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.649356 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.650828 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.651566 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.651723 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:08.652080 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:08.652256 (+ 176us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:08.652354 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:08.652473 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 06:50:08.652559 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260619 06:50:08.661029 6856 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:08.658901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58512 (local address 127.16.79.124:32935)
0619 06:50:08.659149 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.659155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.659571 (+ 416us) server_negotiation.cc:407] Connection header received
0619 06:50:08.659761 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.659765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.659834 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.659917 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:08.660781 (+ 864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58512: BlockingRecv error: recv got EOF from 127.0.0.1:58512 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
I20260619 06:50:08.678194 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:08.672500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56706 (local address 127.16.79.125:38073)
0619 06:50:08.672738 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.672744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.673039 (+ 295us) server_negotiation.cc:407] Connection header received
0619 06:50:08.673210 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.673214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.673279 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.673355 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:08.673786 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.674485 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.676422 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.676871 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.676962 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:08.677429 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:08.677550 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:08.677626 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:08.677781 (+ 155us) server_negotiation.cc:299] Negotiation successful
0619 06:50:08.677842 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
W20260619 06:50:08.686823 6856 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:08.685067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58520 (local address 127.16.79.124:32935)
0619 06:50:08.685242 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.685249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.685537 (+ 288us) server_negotiation.cc:407] Connection header received
0619 06:50:08.685673 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.685678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.685773 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.685878 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:08.686608 (+ 730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58520: BlockingRecv error: recv got EOF from 127.0.0.1:58520 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 06:50:08.712906 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:08.706624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56712 (local address 127.16.79.125:38073)
0619 06:50:08.706847 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.706854 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.707305 (+ 451us) server_negotiation.cc:407] Connection header received
0619 06:50:08.707534 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.707539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.707643 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.707776 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:08.708257 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.709476 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.710805 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.711493 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.711624 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:08.712059 (+ 435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:08.712233 (+ 174us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:08.712328 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:08.712469 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 06:50:08.712581 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260619 06:50:08.721213 6856 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:08.719359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58532 (local address 127.16.79.124:32935)
0619 06:50:08.719537 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.719544 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.719881 (+ 337us) server_negotiation.cc:407] Connection header received
0619 06:50:08.720018 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.720024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.720117 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.720253 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:08.720977 (+ 724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58532: BlockingRecv error: recv got EOF from 127.0.0.1:58532 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 06:50:08.768317 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:08.762194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56728 (local address 127.16.79.125:38073)
0619 06:50:08.762405 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.762412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.762725 (+ 313us) server_negotiation.cc:407] Connection header received
0619 06:50:08.762936 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.762940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.763025 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.763112 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:08.763631 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.764437 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.766464 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.766919 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.767010 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:08.767513 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:08.767628 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:08.767692 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:08.767877 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 06:50:08.767941 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260619 06:50:08.776635 6856 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:08.774517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58548 (local address 127.16.79.124:32935)
0619 06:50:08.774750 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.774757 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.775202 (+ 445us) server_negotiation.cc:407] Connection header received
0619 06:50:08.775443 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.775448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.775521 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.775634 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:08.776365 (+ 731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58548: BlockingRecv error: recv got EOF from 127.0.0.1:58548 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 06:50:08.851563 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:08.844606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56738 (local address 127.16.79.125:38073)
0619 06:50:08.844835 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.844841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.845274 (+ 433us) server_negotiation.cc:407] Connection header received
0619 06:50:08.845512 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.845518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.845611 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.845730 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:08.846292 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.847476 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.849427 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:08.850083 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:08.850216 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:08.850737 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:08.850894 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:08.850983 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:08.851173 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 06:50:08.851261 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260619 06:50:08.870397 6856 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:08.865637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58552 (local address 127.16.79.124:32935)
0619 06:50:08.865814 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:08.865820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:08.868754 (+ 2934us) server_negotiation.cc:407] Connection header received
0619 06:50:08.868980 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:08.868988 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:08.869112 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:08.869256 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:08.870171 (+ 915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58552: BlockingRecv error: recv got EOF from 127.0.0.1:58552 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 06:50:08.946547 6829 catalog_manager.cc:1259] Loaded cluster ID: 196b4d26647d4863b76d7c4f87b4fadf
I20260619 06:50:08.946650 6829 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8: loading cluster ID for follower catalog manager: success
I20260619 06:50:08.949198 6829 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8: acquiring CA information for follower catalog manager: success
I20260619 06:50:08.951050 6829 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c61e661d2c754c73b6ed27b748b00af8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 06:50:09.006585 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:09.001671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56750 (local address 127.16.79.125:38073)
0619 06:50:09.001862 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:09.001867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:09.002207 (+ 340us) server_negotiation.cc:407] Connection header received
0619 06:50:09.002369 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:09.002373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:09.002442 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:09.002530 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:09.002888 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:09.003563 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:09.005080 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:09.005518 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:09.005647 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:09.005967 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:09.006083 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:09.006150 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:09.006318 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 06:50:09.006380 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:50:09.019776 6856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:09.014101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58556 (local address 127.16.79.124:32935)
0619 06:50:09.014342 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:09.014348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:09.014736 (+ 388us) server_negotiation.cc:407] Connection header received
0619 06:50:09.014866 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:09.014870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:09.015024 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:09.015125 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:09.015485 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:09.016284 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:09.017695 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:09.018101 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:09.018196 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:09.018680 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:09.019180 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:09.019289 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:09.019377 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 06:50:09.019442 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 06:50:09.035104 6856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:09.026346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58564 (local address 127.16.79.124:32935)
0619 06:50:09.026596 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:09.026605 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:09.029359 (+ 2754us) server_negotiation.cc:407] Connection header received
0619 06:50:09.029641 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:09.029647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:09.029733 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:09.029802 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:09.030575 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:09.031311 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:09.033087 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:09.033554 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:09.033670 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:09.034250 (+ 580us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:09.034397 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:09.034462 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:09.034678 (+ 216us) server_negotiation.cc:299] Negotiation successful
0619 06:50:09.034761 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260619 06:50:09.035342 6741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:09.026454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56760 (local address 127.16.79.125:38073)
0619 06:50:09.026683 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:09.026691 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:09.029089 (+ 2398us) server_negotiation.cc:407] Connection header received
0619 06:50:09.029378 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:09.029382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:09.029487 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:09.029575 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:09.030367 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:09.031423 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:09.033630 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:09.034037 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:09.034180 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:09.034583 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:09.034721 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:09.034802 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:09.035008 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 06:50:09.035085 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:50:09.036329 6983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:09.026192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36086 (local address 127.16.79.126:40079)
0619 06:50:09.028198 (+ 2006us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:09.028206 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:09.028790 (+ 584us) server_negotiation.cc:407] Connection header received
0619 06:50:09.029052 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:09.029058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:09.029295 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:09.029482 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:09.030030 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:09.031118 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:09.033541 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:09.034181 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:09.034289 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:09.034943 (+ 654us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:09.035412 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:09.035519 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:09.035927 (+ 408us) server_negotiation.cc:299] Negotiation successful
0619 06:50:09.036007 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1850,"thread_start_us":1747,"threads_started":1}
I20260619 06:50:09.041363 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 6595
I20260619 06:50:09.042661 6657 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:09.154076 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 6595
I20260619 06:50:09.175118 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 6669
I20260619 06:50:09.176324 6731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:09.285241 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 6669
I20260619 06:50:09.305294 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 6743
I20260619 06:50:09.306527 6806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:09.414341 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 6743
2026-06-19T06:50:09Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4408 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 06:50:09 dist-test-slave-5w2x krb5kdc[6991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 06:50:09 dist-test-slave-5w2x krb5kdc[6991](info): set up 2 sockets
Jun 19 06:50:09 dist-test-slave-5w2x krb5kdc[6991](info): commencing operation
krb5kdc: starting...
W20260619 06:50:10.520773 16701 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/test-user.keytab.
Jun 19 06:50:10 dist-test-slave-5w2x krb5kdc[6991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T06:50:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T06:50:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.16.79.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
Entry for principal kudu/127.16.79.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab.
I20260619 06:50:10.782822 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:37663
--webserver_interface=127.16.79.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:38499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:37663,127.16.79.125:46147,127.16.79.124:38507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 06:50:11.113243 7004 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 06:50:11.113639 7004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:11.113767 7004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:11.113911 7004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:11.121169 7004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:11.121318 7004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:11.121430 7004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:11.121519 7004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:11.121613 7004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:11.131732 7004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:38499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:37663,127.16.79.125:46147,127.16.79.124:38507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.126.keytab
--principal=kudu/127.16.79.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.126:37663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.16.79.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:11.133158 7004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:11.134500 7004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:11.146505 7010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:11.146528 7009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:11.146564 7012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:11.147694 7004 server_base.cc:1034] running on GCE node
I20260619 06:50:11.148671 7004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:11.150280 7004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:11.151532 7004 hybrid_clock.cc:648] HybridClock initialized: now 1781851811151470 us; error 27 us; skew 500 ppm
I20260619 06:50:11.151934 7004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:11 dist-test-slave-5w2x krb5kdc[6991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851811, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:11.159565 7004 init.cc:377] Logged in from keytab as kudu/127.16.79.126@KRBTEST.COM (short username kudu)
I20260619 06:50:11.161213 7004 webserver.cc:460] Webserver started at http://127.16.79.126:39335/ using document root <none> and password file <none>
I20260619 06:50:11.161952 7004 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:11.162045 7004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:11.162385 7004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:11.165549 7004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/data/instance:
uuid: "2b77da38205f4df895d2b9f6d801a095"
format_stamp: "Formatted at 2026-06-19 06:50:11 on dist-test-slave-5w2x"
server_key: "73aa4b00ef0dddcd20ef6cde275c81ea"
server_key_iv: "f6902fd1f2754cd5ada50cdad31d1e67"
server_key_version: "encryptionkey@0"
I20260619 06:50:11.166407 7004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/wal/instance:
uuid: "2b77da38205f4df895d2b9f6d801a095"
format_stamp: "Formatted at 2026-06-19 06:50:11 on dist-test-slave-5w2x"
server_key: "73aa4b00ef0dddcd20ef6cde275c81ea"
server_key_iv: "f6902fd1f2754cd5ada50cdad31d1e67"
server_key_version: "encryptionkey@0"
I20260619 06:50:11.171743 7004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260619 06:50:11.175451 7019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:11.176970 7004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:50:11.177174 7004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/wal
uuid: "2b77da38205f4df895d2b9f6d801a095"
format_stamp: "Formatted at 2026-06-19 06:50:11 on dist-test-slave-5w2x"
server_key: "73aa4b00ef0dddcd20ef6cde275c81ea"
server_key_iv: "f6902fd1f2754cd5ada50cdad31d1e67"
server_key_version: "encryptionkey@0"
I20260619 06:50:11.177420 7004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:11.206723 7004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:11.210907 7004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:11.211221 7004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:11.230549 7004 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.126:37663
I20260619 06:50:11.230638 7070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.126:37663 every 8 connection(s)
I20260619 06:50:11.231956 7004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/data/info.pb
I20260619 06:50:11.235723 7071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:11.237255 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 7004
I20260619 06:50:11.237462 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-0/wal/instance
I20260619 06:50:11.237895 16701 external_mini_cluster.cc:1428] Setting key 5980612ac527f7e70ac546f40d76abc0
I20260619 06:50:11.242476 7071 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:50:11 dist-test-slave-5w2x krb5kdc[6991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851811, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.79.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.125@KRBTEST.COM" created.
I20260619 06:50:11.269950 7075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.246020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36560 (local address 127.16.79.126:37663)
0619 06:50:11.247926 (+ 1906us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:11.247932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:11.247976 (+ 44us) server_negotiation.cc:407] Connection header received
0619 06:50:11.250308 (+ 2332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:11.250314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:11.250565 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:11.250725 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:11.252443 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:11.253704 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:11.255996 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:11.257033 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:11.257260 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:11.262050 (+ 4790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:11.262064 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:11.262112 (+ 48us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:11.262190 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:11.265620 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:11.266435 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:11.266441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:11.266443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:11.266672 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:11.267134 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:11.267139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:11.267142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:11.267641 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:11.267802 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:11.268535 (+ 733us) server_negotiation.cc:299] Negotiation successful
0619 06:50:11.268871 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":1713,"spinlock_wait_cycles":27392,"thread_start_us":1477,"threads_started":1}
I20260619 06:50:11.269984 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.246020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:37663 (local address 127.0.0.1:36560)
0619 06:50:11.246944 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.246998 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:11.249900 (+ 2902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:11.250799 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:11.250809 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:11.251652 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:11.252173 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:11.252201 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:11.253875 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:11.253884 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:11.255780 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:11.255802 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:11.257301 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:11.257308 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:11.257487 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:11.258217 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:11.258254 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:11.261750 (+ 3496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:11.265764 (+ 4014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:11.265775 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:11.265784 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:11.266311 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:11.266788 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:11.266791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:11.266794 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:11.267003 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:11.267790 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:11.267801 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:11.268064 (+ 263us) client_negotiation.cc:769] Sending connection context
0619 06:50:11.268411 (+ 347us) client_negotiation.cc:240] Negotiation successful
0619 06:50:11.269039 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"mutex_wait_us":71,"thread_start_us":369,"threads_started":1}
I20260619 06:50:11.273029 7071 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:50:11.274241 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.273850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:46147 (local address 127.0.0.1:38616)
0619 06:50:11.274015 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.274121 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:46147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260619 06:50:11.274518 7021 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.79.125:46147: connect: Connection refused (error 111)
I20260619 06:50:11.276175 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.275776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:46147 (local address 127.0.0.1:38622)
0619 06:50:11.275937 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.276003 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:46147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260619 06:50:11.276381 7071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.125:46147: Network error: Client connection negotiation failed: client connection to 127.16.79.125:46147: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.125.keytab.
Entry for principal kudu/127.16.79.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.125.keytab.
I20260619 06:50:11.299376 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.125:46147
--webserver_interface=127.16.79.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.125.keytab
--principal=kudu/127.16.79.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:38499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:37663,127.16.79.125:46147,127.16.79.124:38507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:50:11.325733 7071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } attempt: 1
I20260619 06:50:11.327011 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.326466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:46147 (local address 127.0.0.1:38636)
0619 06:50:11.326684 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.326842 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:46147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260619 06:50:11.328238 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.327809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:46147 (local address 127.0.0.1:38646)
0619 06:50:11.327959 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.328049 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:46147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 06:50:11.328423 7071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.125:46147: Network error: Client connection negotiation failed: client connection to 127.16.79.125:46147: connect: Connection refused (error 111)
I20260619 06:50:11.396699 7071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } attempt: 2
I20260619 06:50:11.398092 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.397434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:46147 (local address 127.0.0.1:38650)
0619 06:50:11.397747 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.397885 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:46147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260619 06:50:11.399251 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.398803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:46147 (local address 127.0.0.1:38662)
0619 06:50:11.398981 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.399092 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:46147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260619 06:50:11.399461 7071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.125:46147: Network error: Client connection negotiation failed: client connection to 127.16.79.125:46147: connect: Connection refused (error 111)
I20260619 06:50:11.490718 7071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } attempt: 3
I20260619 06:50:11.492161 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.491478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:46147 (local address 127.0.0.1:38670)
0619 06:50:11.491721 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.491960 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:46147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260619 06:50:11.493310 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.492876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:46147 (local address 127.0.0.1:38678)
0619 06:50:11.493046 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.493141 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:46147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 06:50:11.493615 7071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.125:46147: Network error: Client connection negotiation failed: client connection to 127.16.79.125:46147: connect: Connection refused (error 111)
I20260619 06:50:11.636927 7071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } attempt: 4
I20260619 06:50:11.638260 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.637698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:46147 (local address 127.0.0.1:38682)
0619 06:50:11.638034 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.638123 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:46147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260619 06:50:11.639288 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.638928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:46147 (local address 127.0.0.1:38694)
0619 06:50:11.639072 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.639168 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.125:46147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 06:50:11.639487 7071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.125:46147: Network error: Client connection negotiation failed: client connection to 127.16.79.125:46147: connect: Connection refused (error 111)
W20260619 06:50:11.659518 7078 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 06:50:11.659922 7078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:11.660055 7078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:11.660234 7078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:11.667408 7078 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:11.667536 7078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:11.667647 7078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:11.667740 7078 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:11.667833 7078 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:11.677814 7078 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:38499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:37663,127.16.79.125:46147,127.16.79.124:38507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.125.keytab
--principal=kudu/127.16.79.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.125:46147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.16.79.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:11.679119 7078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:11.680475 7078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 06:50:11.692054 7083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:11.692220 7084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:11.692229 7086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:11.697698 7078 server_base.cc:1034] running on GCE node
I20260619 06:50:11.698546 7078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:11.700026 7078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:11.701249 7078 hybrid_clock.cc:648] HybridClock initialized: now 1781851811701179 us; error 43 us; skew 500 ppm
I20260619 06:50:11.701606 7078 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:11 dist-test-slave-5w2x krb5kdc[6991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851811, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:11.704866 7078 init.cc:377] Logged in from keytab as kudu/127.16.79.125@KRBTEST.COM (short username kudu)
I20260619 06:50:11.706440 7078 webserver.cc:460] Webserver started at http://127.16.79.125:41071/ using document root <none> and password file <none>
I20260619 06:50:11.707168 7078 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:11.707280 7078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:11.707608 7078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:11.710664 7078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/data/instance:
uuid: "a4888dc6293740aea4fdf05148ba923a"
format_stamp: "Formatted at 2026-06-19 06:50:11 on dist-test-slave-5w2x"
server_key: "3433886316e44a0d5b9810b4b719b085"
server_key_iv: "99e0f045e1e32639dfe1ad5144337d53"
server_key_version: "encryptionkey@0"
I20260619 06:50:11.711444 7078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/wal/instance:
uuid: "a4888dc6293740aea4fdf05148ba923a"
format_stamp: "Formatted at 2026-06-19 06:50:11 on dist-test-slave-5w2x"
server_key: "3433886316e44a0d5b9810b4b719b085"
server_key_iv: "99e0f045e1e32639dfe1ad5144337d53"
server_key_version: "encryptionkey@0"
I20260619 06:50:11.716536 7078 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260619 06:50:11.719882 7093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:11.721223 7078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 06:50:11.721413 7078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/wal
uuid: "a4888dc6293740aea4fdf05148ba923a"
format_stamp: "Formatted at 2026-06-19 06:50:11 on dist-test-slave-5w2x"
server_key: "3433886316e44a0d5b9810b4b719b085"
server_key_iv: "99e0f045e1e32639dfe1ad5144337d53"
server_key_version: "encryptionkey@0"
I20260619 06:50:11.721639 7078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:11.737310 7078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:11.741299 7078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:11.741561 7078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:11.760846 7078 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.125:46147
I20260619 06:50:11.760883 7144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.125:46147 every 8 connection(s)
I20260619 06:50:11.762184 7078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/data/info.pb
I20260619 06:50:11.765596 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 7078
I20260619 06:50:11.765796 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-1/wal/instance
I20260619 06:50:11.766184 7145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:11.766356 16701 external_mini_cluster.cc:1428] Setting key 1e19a2493cce602771b23a9e9d339aaf
I20260619 06:50:11.776036 7145 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:50:11 dist-test-slave-5w2x krb5kdc[6991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851811, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.125@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.16.79.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.16.79.124@KRBTEST.COM" created.
I20260619 06:50:11.799016 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.779027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36564 (local address 127.16.79.126:37663)
0619 06:50:11.779535 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:11.779543 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:11.780594 (+ 1051us) server_negotiation.cc:407] Connection header received
0619 06:50:11.782728 (+ 2134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:11.782737 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:11.782840 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:11.782971 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:11.784568 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:11.785761 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:11.787271 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:11.788050 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:11.788242 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:11.792757 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:11.792767 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:11.792774 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:11.792845 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:11.795740 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:11.796725 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:11.796733 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:11.796742 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:11.796863 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:11.797284 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:11.797293 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:11.797295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:11.797568 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:11.797737 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:11.798425 (+ 688us) server_negotiation.cc:299] Negotiation successful
0619 06:50:11.798642 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":229,"threads_started":1}
I20260619 06:50:11.800010 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.779604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:37663 (local address 127.0.0.1:36564)
0619 06:50:11.780429 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.780475 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:11.782449 (+ 1974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:11.783192 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:11.783200 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:11.783717 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:11.784360 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:11.784380 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:11.785915 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:11.785919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:11.787126 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:11.787138 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:11.788308 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:11.788313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:11.788507 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:11.789263 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:11.789306 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:11.792496 (+ 3190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:11.795917 (+ 3421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:11.795928 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:11.795948 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:11.796585 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:11.796979 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:11.796985 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:11.796988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:11.797162 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:11.797697 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:11.797706 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:11.798059 (+ 353us) client_negotiation.cc:769] Sending connection context
0619 06:50:11.798327 (+ 268us) client_negotiation.cc:240] Negotiation successful
0619 06:50:11.798848 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":325,"threads_started":1}
I20260619 06:50:11.801636 7145 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:50:11 dist-test-slave-5w2x krb5kdc[6991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851811, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.125@KRBTEST.COM for kudu/127.16.79.125@KRBTEST.COM
I20260619 06:50:11.816988 7150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.802510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38696 (local address 127.16.79.125:46147)
0619 06:50:11.803186 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:11.803196 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:11.803229 (+ 33us) server_negotiation.cc:407] Connection header received
0619 06:50:11.803287 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:11.803297 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:11.803540 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:11.803695 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:11.804637 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:11.805698 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:11.807548 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:11.808061 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:11.808227 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:11.811441 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:11.811452 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:11.811465 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:11.811495 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:11.814167 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:11.814797 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:11.814809 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:11.814811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:11.814919 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:11.815264 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:11.815271 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:11.815274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:11.815622 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:11.815865 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:11.816352 (+ 487us) server_negotiation.cc:299] Negotiation successful
0619 06:50:11.816679 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":124,"server-negotiator.queue_time_us":479,"thread_start_us":326,"threads_started":1}
I20260619 06:50:11.817086 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.802520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:46147 (local address 127.0.0.1:38696)
0619 06:50:11.802744 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.802770 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:11.802894 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:11.803692 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:11.803696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:11.804257 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:11.804462 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:11.804476 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:11.805839 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:11.805847 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:11.807409 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:11.807427 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:11.808264 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:11.808270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:11.808444 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:11.809019 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:11.809035 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:11.811241 (+ 2206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:11.814288 (+ 3047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:11.814295 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:11.814299 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:11.814667 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:11.815022 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:11.815028 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:11.815032 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:11.815166 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:11.815827 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:11.815835 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:11.816029 (+ 194us) client_negotiation.cc:769] Sending connection context
0619 06:50:11.816228 (+ 199us) client_negotiation.cc:240] Negotiation successful
0619 06:50:11.816562 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 06:50:11.819687 7145 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:50:11.820889 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.820485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:32984)
0619 06:50:11.820630 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.820738 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260619 06:50:11.821182 7094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
I20260619 06:50:11.822678 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.822394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:32998)
0619 06:50:11.822508 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.822581 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 06:50:11.822868 7145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:38507: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.16.79.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.124.keytab.
Entry for principal kudu/127.16.79.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.124.keytab.
I20260619 06:50:11.827198 16701 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
/tmp/dist-test-taskf8Saqq/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.124:38507
--webserver_interface=127.16.79.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.124.keytab
--principal=kudu/127.16.79.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.16.79.84:38499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.16.79.126:37663,127.16.79.125:46147,127.16.79.124:38507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 06:50:11.872228 7145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } attempt: 1
I20260619 06:50:11.873407 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.872932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33010)
0619 06:50:11.873162 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.873263 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260619 06:50:11.874420 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.874113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33026)
0619 06:50:11.874240 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.874322 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 06:50:11.874688 7145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:38507: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
I20260619 06:50:11.938774 7071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } attempt: 5
I20260619 06:50:11.942883 7145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } attempt: 2
I20260619 06:50:11.943830 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.943389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33036)
0619 06:50:11.943560 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.943652 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 06:50:11.945219 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.944802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33052)
0619 06:50:11.944970 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.945068 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260619 06:50:11.945528 7145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:38507: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Jun 19 06:50:11 dist-test-slave-5w2x krb5kdc[6991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851811, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for kudu/127.16.79.125@KRBTEST.COM
I20260619 06:50:11.953825 7150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.939791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38712 (local address 127.16.79.125:46147)
0619 06:50:11.939975 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:11.939981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:11.940009 (+ 28us) server_negotiation.cc:407] Connection header received
0619 06:50:11.940256 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:11.940261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:11.940344 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:11.940428 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:11.941372 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:11.942260 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:11.943875 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:11.944541 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:11.944703 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:11.948297 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:11.948302 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:11.948305 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:11.948338 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:11.951045 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:11.951782 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:11.951788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:11.951792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:11.951934 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:11.952383 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:11.952389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:11.952392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:11.952696 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:11.952858 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:11.953269 (+ 411us) server_negotiation.cc:299] Negotiation successful
0619 06:50:11.953504 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 06:50:11.953827 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.939667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:46147 (local address 127.0.0.1:38712)
0619 06:50:11.939900 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.939922 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:11.940052 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:11.940501 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:11.940506 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:11.940990 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:11.941221 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:11.941234 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:11.942404 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:11.942409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:11.943709 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:11.943726 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:11.944931 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:11.944940 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:11.945130 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:11.945841 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:11.945866 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:11.948050 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:11.951218 (+ 3168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:11.951224 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:11.951228 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:11.951653 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:11.952059 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:11.952064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:11.952068 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:11.952277 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:11.952820 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:11.952825 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:11.952993 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 06:50:11.953149 (+ 156us) client_negotiation.cc:240] Negotiation successful
0619 06:50:11.953477 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260619 06:50:11.954869 7071 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:50:11.956023 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.955607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33058)
0619 06:50:11.955799 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.955879 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260619 06:50:11.956974 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:11.956676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33070)
0619 06:50:11.956803 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:11.956884 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 06:50:11.957248 7071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:38507: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
I20260619 06:50:12.008468 7071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } attempt: 1
I20260619 06:50:12.009701 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.009242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33084)
0619 06:50:12.009441 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:12.009528 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260619 06:50:12.010670 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.010343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33086)
0619 06:50:12.010484 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:12.010560 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 06:50:12.010875 7071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:38507: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
I20260619 06:50:12.036804 7145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } attempt: 3
I20260619 06:50:12.037931 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.037451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33096)
0619 06:50:12.037685 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:12.037778 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 06:50:12.038987 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.038596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33110)
0619 06:50:12.038811 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:12.038880 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260619 06:50:12.039233 7145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:38507: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
I20260619 06:50:12.079057 7071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } attempt: 2
I20260619 06:50:12.080127 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.079682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33120)
0619 06:50:12.079881 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:12.079972 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 06:50:12.081190 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.080798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33132)
0619 06:50:12.080959 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:12.081030 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 06:50:12.081383 7071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:38507: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
W20260619 06:50:12.157451 7153 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 06:50:12.157847 7153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 06:50:12.157975 7153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 06:50:12.158129 7153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 06:50:12.165565 7153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 06:50:12.165701 7153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 06:50:12.165827 7153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 06:50:12.165910 7153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 06:50:12.166018 7153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 06:50:12.176011 7153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.16.79.84:38499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.16.79.126:37663,127.16.79.125:46147,127.16.79.124:38507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/krb5kdc/kudu_127.16.79.124.keytab
--principal=kudu/127.16.79.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.16.79.124:38507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.16.79.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 19 Jun 2026 05:51:38 UTC on ccycloud.quasar-yggyfi.root.comops.site
build id 4455158
ASAN enabled
I20260619 06:50:12.177380 7153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 06:50:12.178673 7153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 06:50:12.182466 7145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } attempt: 4
I20260619 06:50:12.183619 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.183071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33142)
0619 06:50:12.183328 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:12.183436 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260619 06:50:12.184868 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.184497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33150)
0619 06:50:12.184675 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:12.184767 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260619 06:50:12.185139 7145 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:38507: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
I20260619 06:50:12.187595 7071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } attempt: 3
I20260619 06:50:12.188566 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.188171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33152)
0619 06:50:12.188341 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:12.188427 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 06:50:12.189558 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.189181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33156)
0619 06:50:12.189398 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:12.189461 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260619 06:50:12.189792 7071 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.16.79.124:38507: Network error: Client connection negotiation failed: client connection to 127.16.79.124:38507: connect: Connection refused (error 111)
W20260619 06:50:12.191232 7158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:12.191293 7159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 06:50:12.191778 7161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 06:50:12.192916 7153 server_base.cc:1034] running on GCE node
I20260619 06:50:12.193706 7153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 06:50:12.195115 7153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 06:50:12.196352 7153 hybrid_clock.cc:648] HybridClock initialized: now 1781851812196281 us; error 47 us; skew 500 ppm
I20260619 06:50:12.196698 7153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Jun 19 06:50:12 dist-test-slave-5w2x krb5kdc[6991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851812, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 06:50:12.199826 7153 init.cc:377] Logged in from keytab as kudu/127.16.79.124@KRBTEST.COM (short username kudu)
I20260619 06:50:12.201349 7153 webserver.cc:460] Webserver started at http://127.16.79.124:33559/ using document root <none> and password file <none>
I20260619 06:50:12.201994 7153 fs_manager.cc:359] Metadata directory not provided
I20260619 06:50:12.202080 7153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 06:50:12.202337 7153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 06:50:12.205302 7153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/data/instance:
uuid: "64c0dbf9e9e94a7db56416370f97b2f7"
format_stamp: "Formatted at 2026-06-19 06:50:12 on dist-test-slave-5w2x"
server_key: "c57c9c1d3f7d6e79702123a2c10cfcf0"
server_key_iv: "385a02e4e5d6d28690d91df8301cd1e4"
server_key_version: "encryptionkey@0"
I20260619 06:50:12.206017 7153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/wal/instance:
uuid: "64c0dbf9e9e94a7db56416370f97b2f7"
format_stamp: "Formatted at 2026-06-19 06:50:12 on dist-test-slave-5w2x"
server_key: "c57c9c1d3f7d6e79702123a2c10cfcf0"
server_key_iv: "385a02e4e5d6d28690d91df8301cd1e4"
server_key_version: "encryptionkey@0"
I20260619 06:50:12.210933 7153 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 06:50:12.214354 7168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 06:50:12.215636 7153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 06:50:12.215798 7153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/data,/tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/wal
uuid: "64c0dbf9e9e94a7db56416370f97b2f7"
format_stamp: "Formatted at 2026-06-19 06:50:12 on dist-test-slave-5w2x"
server_key: "c57c9c1d3f7d6e79702123a2c10cfcf0"
server_key_iv: "385a02e4e5d6d28690d91df8301cd1e4"
server_key_version: "encryptionkey@0"
I20260619 06:50:12.216018 7153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 06:50:12.227011 7153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 06:50:12.231498 7153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 06:50:12.231767 7153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 06:50:12.250519 7153 rpc_server.cc:307] RPC server started. Bound to: 127.16.79.124:38507
I20260619 06:50:12.250540 7219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.16.79.124:38507 every 8 connection(s)
I20260619 06:50:12.252000 7153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/data/info.pb
I20260619 06:50:12.255658 7220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 06:50:12.261413 16701 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu as pid 7153
I20260619 06:50:12.261605 16701 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskf8Saqq/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781851451399607-16701-0/minicluster-data/master-2/wal/instance
I20260619 06:50:12.262019 16701 external_mini_cluster.cc:1428] Setting key ef56b637155744535a0b0988eb26d6da
I20260619 06:50:12.262646 7220 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:50:12 dist-test-slave-5w2x krb5kdc[6991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851812, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.124@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
Jun 19 06:50:12 dist-test-slave-5w2x krb5kdc[6991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.124@KRBTEST.COM
I20260619 06:50:12.283463 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.265439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36578 (local address 127.16.79.126:37663)
0619 06:50:12.265639 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.265647 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.266469 (+ 822us) server_negotiation.cc:407] Connection header received
0619 06:50:12.266883 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.266889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.267013 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.267122 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.268855 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.269950 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.272002 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.272620 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.272747 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:12.276782 (+ 4035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:12.276787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:12.276791 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:12.276825 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:12.280152 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.280996 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.281003 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.281007 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.281150 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.281620 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.281626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.281631 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.282027 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:12.282178 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:12.282897 (+ 719us) server_negotiation.cc:299] Negotiation successful
0619 06:50:12.283141 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 06:50:12.284600 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.265469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.126:37663 (local address 127.0.0.1:36578)
0619 06:50:12.266311 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:12.266358 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:12.266662 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:12.267236 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:12.267243 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:12.268099 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:12.268672 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:12.268696 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.270109 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.270118 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:12.271843 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:12.271863 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.272818 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.272823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:12.272980 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:12.273719 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:12.273771 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:12.276599 (+ 2828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:12.280290 (+ 3691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:12.280299 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:12.280307 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:12.280878 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:12.281310 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:12.281313 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:12.281315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:12.281486 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:12.282157 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:12.282164 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:12.282539 (+ 375us) client_negotiation.cc:769] Sending connection context
0619 06:50:12.282813 (+ 274us) client_negotiation.cc:240] Negotiation successful
0619 06:50:12.283297 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":391,"threads_started":1}
I20260619 06:50:12.284578 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.263788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33160 (local address 127.16.79.124:38507)
0619 06:50:12.264576 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.264586 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.264626 (+ 40us) server_negotiation.cc:407] Connection header received
0619 06:50:12.266592 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.266612 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.266892 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.267110 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.268866 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.270421 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.272192 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.272993 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.273081 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:12.277546 (+ 4465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:12.277559 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:12.277578 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:12.277625 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:12.281134 (+ 3509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.281921 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.281929 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.281935 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.282073 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.282482 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.282490 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.282493 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.282873 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:12.283178 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:12.283494 (+ 316us) server_negotiation.cc:299] Negotiation successful
0619 06:50:12.283744 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"spinlock_wait_cycles":29056,"thread_start_us":360,"threads_started":1}
I20260619 06:50:12.286259 7220 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 06:50:12 dist-test-slave-5w2x krb5kdc[6991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851812, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.124@KRBTEST.COM for kudu/127.16.79.125@KRBTEST.COM
Jun 19 06:50:12 dist-test-slave-5w2x krb5kdc[6991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.126@KRBTEST.COM
I20260619 06:50:12.300127 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.287119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.125:46147 (local address 127.0.0.1:38716)
0619 06:50:12.287394 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:12.287413 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:12.287508 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:12.287813 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:12.287817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:12.288292 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:12.288481 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:12.288492 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.289402 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.289408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:12.290424 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:12.290438 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.291088 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.291095 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:12.291301 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:12.291891 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:12.291908 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:12.294308 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:12.297970 (+ 3662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:12.297974 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:12.297976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:12.298312 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:12.298746 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:12.298750 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:12.298752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:12.298886 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:12.299474 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:12.299476 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:12.299592 (+ 116us) client_negotiation.cc:769] Sending connection context
0619 06:50:12.299710 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 06:50:12.299895 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260619 06:50:12.300457 7150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.287082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38716 (local address 127.16.79.125:46147)
0619 06:50:12.287488 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.287492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.287509 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:50:12.287620 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.287624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.287681 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.287792 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.288607 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.289266 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.290556 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.290952 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.291041 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:12.294518 (+ 3477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:12.294523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:12.294526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:12.294558 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:12.297843 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.298430 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.298437 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.298440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.298634 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.298997 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.299002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.299005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.299360 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:12.299514 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:12.299851 (+ 337us) server_negotiation.cc:299] Negotiation successful
0619 06:50:12.300070 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303}
I20260619 06:50:12.301335 7220 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } has no permanent_uuid. Determining permanent_uuid...
I20260619 06:50:12.303031 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.287964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36584 (local address 127.16.79.126:37663)
0619 06:50:12.288114 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.288117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.288199 (+ 82us) server_negotiation.cc:407] Connection header received
0619 06:50:12.288330 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.288333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.288394 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.288462 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.289665 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.290319 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.291787 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.292283 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.292373 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:12.296902 (+ 4529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:12.296908 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:12.296913 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:12.296957 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:12.300118 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.301030 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.301038 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.301043 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.301200 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.301603 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.301607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.301609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.301962 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:12.302267 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:12.302572 (+ 305us) server_negotiation.cc:299] Negotiation successful
0619 06:50:12.302804 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Jun 19 06:50:12 dist-test-slave-5w2x krb5kdc[6991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851812, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.124@KRBTEST.COM for kudu/127.16.79.124@KRBTEST.COM
Jun 19 06:50:12 dist-test-slave-5w2x krb5kdc[6991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.16.79.125@KRBTEST.COM
I20260619 06:50:12.315328 7224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.302049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33176)
0619 06:50:12.302237 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:12.302255 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:12.302354 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:12.302674 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:12.302681 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:12.303347 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:12.303495 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:12.303503 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.304584 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.304589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:12.305500 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:12.305512 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.306217 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.306222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:12.306518 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:12.307083 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:12.307100 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:12.309318 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:12.312928 (+ 3610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:12.312934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:12.312938 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:12.313299 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:12.313691 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:12.313699 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:12.313705 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:12.313833 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:12.314411 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:12.314415 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:12.314534 (+ 119us) client_negotiation.cc:769] Sending connection context
0619 06:50:12.314656 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 06:50:12.314938 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":35}
I20260619 06:50:12.315560 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.302253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33176 (local address 127.16.79.124:38507)
0619 06:50:12.302386 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.302390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.302402 (+ 12us) server_negotiation.cc:407] Connection header received
0619 06:50:12.302462 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.302469 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.302532 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.302647 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.303639 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.304430 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.305662 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.306069 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.306210 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:12.309519 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:12.309527 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:12.309533 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:12.309568 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:12.312778 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.313428 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.313432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.313436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.313582 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.313943 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.313948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.313951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.314288 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:12.314441 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:12.314788 (+ 347us) server_negotiation.cc:299] Negotiation successful
0619 06:50:12.315146 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":51}
I20260619 06:50:12.320814 7150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.304668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38726 (local address 127.16.79.125:46147)
0619 06:50:12.304829 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.304834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.304851 (+ 17us) server_negotiation.cc:407] Connection header received
0619 06:50:12.305019 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.305024 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.305089 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.305193 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.306360 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.307296 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.308573 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.309216 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.309357 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:12.314240 (+ 4883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:12.314247 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:12.314252 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:12.314288 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:12.317816 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.318639 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.318643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.318645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.318805 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.319238 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.319242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.319244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.319580 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:12.319808 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:12.320258 (+ 450us) server_negotiation.cc:299] Negotiation successful
0619 06:50:12.320477 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 06:50:12.321245 7220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7: Bootstrap starting.
I20260619 06:50:12.325132 7220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:12.325773 16701 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 06:50:12.326336 7220 log.cc:824] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:12.329206 7220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7: No bootstrap required, opened a new log
I20260619 06:50:12.328536 7046 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2b77da38205f4df895d2b9f6d801a095"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36584
I20260619 06:50:12.336369 7220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } }
I20260619 06:50:12.336730 7220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64c0dbf9e9e94a7db56416370f97b2f7, State: Initialized, Role: FOLLOWER
I20260619 06:50:12.337169 7220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } }
I20260619 06:50:12.338020 7226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } } }
I20260619 06:50:12.338332 7226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:50:12.339329 7220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 06:50:12.348582 7237 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:50:12.348706 7237 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:50:12.346448 7153 master_runner.cc:186] Error getting master registration for 127.16.79.126:37663: OK, instance_id { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" instance_seqno: 1781851811225126 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 06:50:12.354593 7153 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 06:50:12.367000 7071 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } attempt: 4
Jun 19 06:50:12 dist-test-slave-5w2x krb5kdc[6991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851811, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.126@KRBTEST.COM for kudu/127.16.79.124@KRBTEST.COM
I20260619 06:50:12.384030 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.367807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33184)
0619 06:50:12.368004 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:12.368025 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:12.368191 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:12.368566 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:12.368573 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:12.369096 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:12.369301 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:12.369312 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.370806 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.370812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:12.372370 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:12.372385 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.373466 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.373474 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:12.373690 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:12.374403 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:12.374474 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:12.377194 (+ 2720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:12.381332 (+ 4138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:12.381338 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:12.381342 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:12.381834 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:12.382255 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:12.382261 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:12.382265 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:12.382441 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:12.383042 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:12.383047 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:12.383216 (+ 169us) client_negotiation.cc:769] Sending connection context
0619 06:50:12.383351 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 06:50:12.383636 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 06:50:12.384035 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.367887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33184 (local address 127.16.79.124:38507)
0619 06:50:12.368102 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.368108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.368181 (+ 73us) server_negotiation.cc:407] Connection header received
0619 06:50:12.368309 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.368318 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.368408 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.368518 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.369468 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.370636 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.372548 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.373296 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.373436 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:12.377399 (+ 3963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:12.377407 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:12.377412 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:12.377453 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:12.381197 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.381971 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.381977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.381990 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.382144 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.382554 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.382559 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.382562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.382919 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:12.383091 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:12.383453 (+ 362us) server_negotiation.cc:299] Negotiation successful
0619 06:50:12.383676 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 06:50:12.388763 7071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095: Bootstrap starting.
I20260619 06:50:12.391706 7071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:12.392720 7071 log.cc:824] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:12.395156 7071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095: No bootstrap required, opened a new log
I20260619 06:50:12.400020 7071 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } }
I20260619 06:50:12.400390 7071 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b77da38205f4df895d2b9f6d801a095, State: Initialized, Role: FOLLOWER
I20260619 06:50:12.400751 7071 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } }
I20260619 06:50:12.401664 7240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } } }
I20260619 06:50:12.402027 7240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:50:12.402705 7071 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:12.402864 7046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 06:50:12.403103 7046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:12.406801 7046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } }
I20260619 06:50:12.410385 7195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b77da38205f4df895d2b9f6d801a095" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "64c0dbf9e9e94a7db56416370f97b2f7"
I20260619 06:50:12.410854 7195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7 [term 0 FOLLOWER]: Advancing to term 1
W20260619 06:50:12.413432 7251 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 06:50:12.413547 7251 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 06:50:12.413962 7046 leader_election.cc:288] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [CANDIDATE]: Term 1 election: Requested vote from peers a4888dc6293740aea4fdf05148ba923a (127.16.79.125:46147), 64c0dbf9e9e94a7db56416370f97b2f7 (127.16.79.124:38507)
I20260619 06:50:12.415987 7195 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b77da38205f4df895d2b9f6d801a095 in term 1.
I20260619 06:50:12.419931 7020 leader_election.cc:302] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b77da38205f4df895d2b9f6d801a095, 64c0dbf9e9e94a7db56416370f97b2f7; no voters:
I20260619 06:50:12.420418 7240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 06:50:12.415191 7004 master_runner.cc:186] Error getting master registration for 127.16.79.125:46147: OK, instance_id { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" instance_seqno: 1781851811755398 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 06:50:12.435293 7004 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 06:50:12.437019 7240 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [term 1 LEADER]: Becoming Leader. State: Replica: 2b77da38205f4df895d2b9f6d801a095, State: Running, Role: LEADER
I20260619 06:50:12.437667 7240 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } }
I20260619 06:50:12.444635 7254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b77da38205f4df895d2b9f6d801a095. Latest consensus state: current_term: 1 leader_uuid: "2b77da38205f4df895d2b9f6d801a095" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } } }
I20260619 06:50:12.446185 7254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:12.454413 7257 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 06:50:12.456693 7257 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260619 06:50:12.464214 7025 tablet.cc:2369] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:50:12.470678 7195 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7 [term 1 FOLLOWER]: Refusing update from remote peer 2b77da38205f4df895d2b9f6d801a095: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:12.471876 7254 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 06:50:12.483930 7174 tablet.cc:2369] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 06:50:12.484339 7145 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } attempt: 5
I20260619 06:50:12.505888 7226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b77da38205f4df895d2b9f6d801a095. Latest consensus state: current_term: 1 leader_uuid: "2b77da38205f4df895d2b9f6d801a095" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } } }
I20260619 06:50:12.506244 7226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:50:12.514338 7260 mvcc.cc:205] Tried to move back new op lower bound from 7298465023865716736 to 7298465023751028736. Current Snapshot: MvccSnapshot[applied={T|T < 7298465023865716736}]
I20260619 06:50:12.514391 7263 mvcc.cc:205] Tried to move back new op lower bound from 7298465023865716736 to 7298465023751028736. Current Snapshot: MvccSnapshot[applied={T|T < 7298465023865716736}]
I20260619 06:50:12.514588 7240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b77da38205f4df895d2b9f6d801a095" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } } }
I20260619 06:50:12.514911 7240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:12.518311 7257 catalog_manager.cc:1347] Generated new cluster ID: d75db5d9f87644649384df18326fb59f
I20260619 06:50:12.518424 7257 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 06:50:12 dist-test-slave-5w2x krb5kdc[6991](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781851811, etypes {rep=17 tkt=17 ses=17}, kudu/127.16.79.125@KRBTEST.COM for kudu/127.16.79.124@KRBTEST.COM
I20260619 06:50:12.524282 7150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.470492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38742 (local address 127.16.79.125:46147)
0619 06:50:12.470678 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.470685 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.484736 (+ 14051us) server_negotiation.cc:407] Connection header received
0619 06:50:12.492266 (+ 7530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.492275 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.492375 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.492466 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.494899 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.495990 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.497863 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.498608 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.498713 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:12.516641 (+ 17928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:12.516651 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:12.516655 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:12.516707 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:12.520837 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.521686 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.521692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.521695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.521872 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.522271 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.522276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.522279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.522605 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:12.522819 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:12.523706 (+ 887us) server_negotiation.cc:299] Negotiation successful
0619 06:50:12.523918 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 06:50:12.534497 7226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b77da38205f4df895d2b9f6d801a095" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } } }
I20260619 06:50:12.534775 7226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:50:12.539083 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.477059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33190 (local address 127.16.79.124:38507)
0619 06:50:12.477238 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.477245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.488419 (+ 11174us) server_negotiation.cc:407] Connection header received
0619 06:50:12.488890 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.488898 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.489069 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.489159 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.492080 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.493272 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.495580 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.496316 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.496428 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:12.517384 (+ 20956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:12.517393 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:12.517399 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:12.517443 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:12.527405 (+ 9962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.533648 (+ 6243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.533654 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.533658 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.533847 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.535258 (+ 1411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.535264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.535268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.535649 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:12.535902 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:12.538468 (+ 2566us) server_negotiation.cc:299] Negotiation successful
0619 06:50:12.538702 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 06:50:12.539608 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.477401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36586 (local address 127.16.79.126:37663)
0619 06:50:12.477558 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.477564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.493499 (+ 15935us) server_negotiation.cc:407] Connection header received
0619 06:50:12.494533 (+ 1034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.494539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.494636 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.494714 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.517730 (+ 23016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.518890 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.522417 (+ 3527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.523200 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.523301 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:12.529839 (+ 6538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:12.529848 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:12.529858 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:12.529900 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:12.533950 (+ 4050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.535773 (+ 1823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.535780 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.535784 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.535956 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.536640 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.536645 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.536649 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.537020 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:12.537288 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:12.538179 (+ 891us) server_negotiation.cc:299] Negotiation successful
0619 06:50:12.538933 (+ 754us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 06:50:12.560846 7149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.484998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.16.79.124:38507 (local address 127.0.0.1:33194)
0619 06:50:12.485161 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 06:50:12.485182 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 06:50:12.485297 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 06:50:12.499757 (+ 14460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 06:50:12.499763 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 06:50:12.500257 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0619 06:50:12.500441 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:12.500452 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.502075 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.502081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:12.503443 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 06:50:12.503458 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.519824 (+ 16366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.519831 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 06:50:12.520103 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:12.520738 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 06:50:12.520762 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 06:50:12.523507 (+ 2745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 06:50:12.556612 (+ 33105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:12.556619 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:12.556623 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:12.557233 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:12.557679 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 06:50:12.557684 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 06:50:12.557688 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 06:50:12.557851 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 06:50:12.559761 (+ 1910us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 06:50:12.559767 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 06:50:12.559928 (+ 161us) client_negotiation.cc:769] Sending connection context
0619 06:50:12.560094 (+ 166us) client_negotiation.cc:240] Negotiation successful
0619 06:50:12.560445 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 06:50:12.560930 7265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:12.485374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33194 (local address 127.16.79.124:38507)
0619 06:50:12.499472 (+ 14098us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.499477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.499497 (+ 20us) server_negotiation.cc:407] Connection header received
0619 06:50:12.499541 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.499545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.499606 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.499705 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.500600 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.501933 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.506891 (+ 4958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:12.513829 (+ 6938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:12.523694 (+ 9865us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:12.523939 (+ 245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 06:50:12.523945 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 06:50:12.523951 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 06:50:12.523987 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 06:50:12.556390 (+ 32403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.557385 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.557390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.557393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.557541 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 06:50:12.559127 (+ 1586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 06:50:12.559132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 06:50:12.559136 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 06:50:12.559542 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 06:50:12.559680 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:12.560243 (+ 563us) server_negotiation.cc:299] Negotiation successful
0619 06:50:12.560476 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13967,"thread_start_us":14734,"threads_started":1}
I20260619 06:50:12.566674 7145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a: Bootstrap starting.
I20260619 06:50:12.576624 7257 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 06:50:12.578492 7257 catalog_manager.cc:1504] Loading token signing keys...
I20260619 06:50:12.582706 7145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a: Neither blocks nor log segments found. Creating new log.
I20260619 06:50:12.585237 7145 log.cc:824] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a: Log is configured to *not* fsync() on all Append() calls
I20260619 06:50:12.593238 7257 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095: Generated new TSK 0
I20260619 06:50:12.594420 7257 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 06:50:12.620630 7145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a: No bootstrap required, opened a new log
I20260619 06:50:12.643064 7145 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } }
I20260619 06:50:12.643734 7145 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4888dc6293740aea4fdf05148ba923a, State: Initialized, Role: FOLLOWER
I20260619 06:50:12.644559 7145 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } }
I20260619 06:50:12.661505 7119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a [term 0 FOLLOWER]: Advancing to term 1
I20260619 06:50:12.664225 7120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2b77da38205f4df895d2b9f6d801a095" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a4888dc6293740aea4fdf05148ba923a"
I20260619 06:50:12.665233 7119 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a [term 1 FOLLOWER]: Refusing update from remote peer 2b77da38205f4df895d2b9f6d801a095: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 06:50:12.666028 7145 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 06:50:12.668798 7120 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b77da38205f4df895d2b9f6d801a095 in term 1.
I20260619 06:50:12.669514 7240 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 06:50:12.672212 7271 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } } }
I20260619 06:50:12.672622 7271 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:50:12.703390 7278 mvcc.cc:205] Tried to move back new op lower bound from 7298465023865716736 to 7298465023751028736. Current Snapshot: MvccSnapshot[applied={T|T < 7298465023865716736}]
W20260619 06:50:12.709254 16701 mini_kdc.cc:348] Time spent kdestroy: real 0.106s user 0.000s sys 0.022s
I20260619 06:50:12.712680 7271 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b77da38205f4df895d2b9f6d801a095. Latest consensus state: current_term: 1 leader_uuid: "2b77da38205f4df895d2b9f6d801a095" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } } }
I20260619 06:50:12.713034 7271 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:50:12.712749 7275 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b77da38205f4df895d2b9f6d801a095" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } } }
I20260619 06:50:12.713141 7275 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a [sys.catalog]: This master's current role is: FOLLOWER
I20260619 06:50:12.717913 7289 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260619 06:50:12.744391 7150 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:12.734156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38754 (local address 127.16.79.125:46147)
0619 06:50:12.734355 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.734360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.741974 (+ 7614us) server_negotiation.cc:407] Connection header received
0619 06:50:12.742518 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.742527 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.742634 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.742713 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.744061 (+ 1348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38754: BlockingRecv error: recv got EOF from 127.0.0.1:38754 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260619 06:50:12.746904 7265 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:12.740846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33202 (local address 127.16.79.124:38507)
0619 06:50:12.741069 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.741079 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.745191 (+ 4112us) server_negotiation.cc:407] Connection header received
0619 06:50:12.745538 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.745546 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.745675 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.745773 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.746663 (+ 890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33202: BlockingRecv error: recv got EOF from 127.0.0.1:33202 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260619 06:50:12.746950 7148 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:12.733697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36592 (local address 127.16.79.126:37663)
0619 06:50:12.733887 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.733893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.744724 (+ 10831us) server_negotiation.cc:407] Connection header received
0619 06:50:12.745037 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.745043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.745138 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.745220 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.746676 (+ 1456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36592: BlockingRecv error: recv got EOF from 127.0.0.1:36592 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260619 06:50:12.753774 7150 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:12.751900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38762 (local address 127.16.79.125:46147)
0619 06:50:12.752077 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.752083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.752105 (+ 22us) server_negotiation.cc:407] Connection header received
0619 06:50:12.752465 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.752471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.752615 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.752727 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.753530 (+ 803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38762: BlockingRecv error: recv got EOF from 127.0.0.1:38762 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260619 06:50:12.773226 7150 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:12.758457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38766 (local address 127.16.79.125:46147)
0619 06:50:12.758621 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.758627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.758814 (+ 187us) server_negotiation.cc:407] Connection header received
0619 06:50:12.759023 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.759029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.759120 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.759225 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.760163 (+ 938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38766: BlockingRecv error: recv got EOF from 127.0.0.1:38766 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260619 06:50:12.800243 7150 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:12.797847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38780 (local address 127.16.79.125:46147)
0619 06:50:12.798037 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.798044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.798580 (+ 536us) server_negotiation.cc:407] Connection header received
0619 06:50:12.798783 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.798791 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.798938 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.799062 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.799925 (+ 863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38780: BlockingRecv error: recv got EOF from 127.0.0.1:38780 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260619 06:50:12.816807 7150 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:12.814545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38784 (local address 127.16.79.125:46147)
0619 06:50:12.814777 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.814784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.814807 (+ 23us) server_negotiation.cc:407] Connection header received
0619 06:50:12.815034 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.815040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.815151 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.815230 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.816562 (+ 1332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38784: BlockingRecv error: recv got EOF from 127.0.0.1:38784 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
W20260619 06:50:12.830346 7150 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:12.828070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38792 (local address 127.16.79.125:46147)
0619 06:50:12.828291 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.828297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.828796 (+ 499us) server_negotiation.cc:407] Connection header received
0619 06:50:12.828996 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.829003 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.829105 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.829223 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.830106 (+ 883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38792: BlockingRecv error: recv got EOF from 127.0.0.1:38792 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260619 06:50:12.852288 7150 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:12.850053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38794 (local address 127.16.79.125:46147)
0619 06:50:12.850239 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.850245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.850792 (+ 547us) server_negotiation.cc:407] Connection header received
0619 06:50:12.850975 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.850981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.851087 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.851199 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.852031 (+ 832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38794: BlockingRecv error: recv got EOF from 127.0.0.1:38794 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260619 06:50:12.890405 7150 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:12.888044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38798 (local address 127.16.79.125:46147)
0619 06:50:12.888250 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.888256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.888728 (+ 472us) server_negotiation.cc:407] Connection header received
0619 06:50:12.889024 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.889030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.889131 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.889260 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.890135 (+ 875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38798: BlockingRecv error: recv got EOF from 127.0.0.1:38798 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260619 06:50:12.959906 7150 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:12.957717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38808 (local address 127.16.79.125:46147)
0619 06:50:12.957946 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:12.957953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:12.958303 (+ 350us) server_negotiation.cc:407] Connection header received
0619 06:50:12.958467 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:12.958474 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:12.958575 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:12.958706 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:12.959677 (+ 971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38808: BlockingRecv error: recv got EOF from 127.0.0.1:38808 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260619 06:50:13.093001 7150 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:13.090962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38820 (local address 127.16.79.125:46147)
0619 06:50:13.091220 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:13.091227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:13.091533 (+ 306us) server_negotiation.cc:407] Connection header received
0619 06:50:13.091727 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:13.091732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:13.091810 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:13.091902 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:13.092770 (+ 868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38820: BlockingRecv error: recv got EOF from 127.0.0.1:38820 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
I20260619 06:50:13.099699 7240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b77da38205f4df895d2b9f6d801a095" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b77da38205f4df895d2b9f6d801a095" member_type: VOTER last_known_addr { host: "127.16.79.126" port: 37663 } } peers { permanent_uuid: "a4888dc6293740aea4fdf05148ba923a" member_type: VOTER last_known_addr { host: "127.16.79.125" port: 46147 } } peers { permanent_uuid: "64c0dbf9e9e94a7db56416370f97b2f7" member_type: VOTER last_known_addr { host: "127.16.79.124" port: 38507 } } }
I20260619 06:50:13.100031 7240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b77da38205f4df895d2b9f6d801a095 [sys.catalog]: This master's current role is: LEADER
I20260619 06:50:13.350720 7237 catalog_manager.cc:1259] Loaded cluster ID: d75db5d9f87644649384df18326fb59f
I20260619 06:50:13.350852 7237 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7: loading cluster ID for follower catalog manager: success
I20260619 06:50:13.353899 7237 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7: acquiring CA information for follower catalog manager: success
W20260619 06:50:13.354497 7150 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 06:50:13.352484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38828 (local address 127.16.79.125:46147)
0619 06:50:13.352691 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:13.352700 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:13.353040 (+ 340us) server_negotiation.cc:407] Connection header received
0619 06:50:13.353292 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:13.353297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:13.353373 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:13.353484 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 06:50:13.354259 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38828: BlockingRecv error: recv got EOF from 127.0.0.1:38828 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 06:50:13.356109 7237 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 64c0dbf9e9e94a7db56416370f97b2f7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 06:50:13.722218 7289 catalog_manager.cc:1259] Loaded cluster ID: d75db5d9f87644649384df18326fb59f
I20260619 06:50:13.722319 7289 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a: loading cluster ID for follower catalog manager: success
I20260619 06:50:13.725026 7289 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a: acquiring CA information for follower catalog manager: success
I20260619 06:50:13.726976 7289 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a4888dc6293740aea4fdf05148ba923a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 06:50:13.890326 7355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:13.883188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38840 (local address 127.16.79.125:46147)
0619 06:50:13.884067 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:13.884073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:13.884089 (+ 16us) server_negotiation.cc:407] Connection header received
0619 06:50:13.884206 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:13.884212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:13.884378 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:13.884523 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:13.884985 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:13.885958 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:13.887871 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:13.888397 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:13.888496 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:13.889111 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:13.889660 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:13.889771 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:13.889910 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 06:50:13.889977 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":525,"threads_started":1}
I20260619 06:50:13.906479 7361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:13.896674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33214 (local address 127.16.79.124:38507)
0619 06:50:13.900487 (+ 3813us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:13.900493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:13.900512 (+ 19us) server_negotiation.cc:407] Connection header received
0619 06:50:13.900627 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:13.900633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:13.900858 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:13.901029 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:13.901387 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:13.902500 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:13.903892 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:13.904499 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:13.904621 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:13.905210 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:13.905853 (+ 643us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:13.906008 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:13.906089 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 06:50:13.906175 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3689,"thread_start_us":3562,"threads_started":1}
I20260619 06:50:13.931509 7361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:13.913009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33224 (local address 127.16.79.124:38507)
0619 06:50:13.913188 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:13.913194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:13.924279 (+ 11085us) server_negotiation.cc:407] Connection header received
0619 06:50:13.924476 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:13.924484 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:13.924603 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:13.924708 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:13.925511 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:13.926702 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:13.929008 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:13.929656 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:13.929753 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:13.930578 (+ 825us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:13.930757 (+ 179us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:13.930841 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:13.931101 (+ 260us) server_negotiation.cc:299] Negotiation successful
0619 06:50:13.931205 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 06:50:13.935108 7366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:13.912562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36596 (local address 127.16.79.126:37663)
0619 06:50:13.924956 (+ 12394us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:13.925008 (+ 52us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:13.925036 (+ 28us) server_negotiation.cc:407] Connection header received
0619 06:50:13.925211 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:13.925219 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:13.925482 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:13.925698 (+ 216us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:13.926775 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:13.928195 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:13.932097 (+ 3902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:13.932893 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:13.933070 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:13.933905 (+ 835us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:13.934270 (+ 365us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:13.934406 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:13.934598 (+ 192us) server_negotiation.cc:299] Negotiation successful
0619 06:50:13.934687 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12137,"thread_start_us":17795,"threads_started":1}
I20260619 06:50:13.938524 7355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 06:50:13.912823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38854 (local address 127.16.79.125:46147)
0619 06:50:13.913104 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0619 06:50:13.913117 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 06:50:13.931754 (+ 18637us) server_negotiation.cc:407] Connection header received
0619 06:50:13.932077 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 06:50:13.932083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 06:50:13.932216 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 06:50:13.932305 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 06:50:13.932886 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:13.934126 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:13.936314 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 06:50:13.936877 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 06:50:13.936996 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 06:50:13.937525 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 06:50:13.937664 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 06:50:13.937739 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 06:50:13.937923 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 06:50:13.938009 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 06:50:13.940196 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 7004
I20260619 06:50:13.941197 7066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:14.065059 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 7004
W20260619 06:50:14.085392 7169 connection.cc:535] server connection from 127.0.0.1:33184 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 06:50:14.085835 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 7078
I20260619 06:50:14.086970 7140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:14.192312 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 7078
I20260619 06:50:14.211807 16701 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu pid 7153
I20260619 06:50:14.212920 7215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260619 06:50:14.318936 16701 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskf8Saqq/build/asan/bin/kudu with pid 7153
2026-06-19T06:50:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4900 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9309 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (362884 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST